Sep 11 13:46:52.520050 ip-10-0-128-161 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 13:46:52.520062 ip-10-0-128-161 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 13:46:52.520070 ip-10-0-128-161 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 13:46:52.520396 ip-10-0-128-161 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 13:47:02.549674 ip-10-0-128-161 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Sep 11 13:47:02.549686 ip-10-0-128-161 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot f3fe2fb507fa480882f4ee0ee599659e --
Sep 11 13:48:38.485994 ip-10-0-128-161 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 13:48:38.917913 ip-10-0-128-161 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:48:38.917913 ip-10-0-128-161 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 13:48:38.917913 ip-10-0-128-161 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:48:38.917913 ip-10-0-128-161 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 13:48:38.917913 ip-10-0-128-161 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:48:38.919069 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.918976    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 13:48:38.925542 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925506    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:48:38.925542 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925533    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:48:38.925542 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925538    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:48:38.925542 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925541    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:48:38.925542 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925544    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:48:38.925542 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925546    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:48:38.925542 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925549    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:48:38.925542 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925552    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:48:38.925542 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925555    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:48:38.925542 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925558    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925562    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925567    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925570    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925574    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925576    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925579    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925582    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925584    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925587    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925591    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925596    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925599    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925602    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925605    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925607    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925610    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925612    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925615    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:48:38.925922 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925618    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925620    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925637    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925647    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925650    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925653    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925655    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925658    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925661    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925663    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925665    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925669    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925671    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925673    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925676    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925679    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925682    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925685    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925688    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925691    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:48:38.926372 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925693    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925696    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925705    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925708    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925710    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925713    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925716    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925718    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925721    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925723    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925725    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925728    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925730    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925733    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925736    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925738    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925741    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925743    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925749    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:48:38.926939 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925753    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925755    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925758    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925760    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925763    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925766    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925768    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925771    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925775    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925778    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925780    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925783    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925785    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925788    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925790    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925793    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925800    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925803    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925806    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925809    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:48:38.927406 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925811    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925814    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925816    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925826    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925829    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925831    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925834    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925836    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925839    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925841    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925844    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.925847    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926703    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926713    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926717    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926720    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926723    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926726    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926728    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926731    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:48:38.927931 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926738    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926741    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926743    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926747    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926749    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926752    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926754    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926757    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926759    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926769    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926772    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926774    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926777    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926780    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926782    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926785    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926788    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926790    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926793    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926795    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:48:38.928418 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926798    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926801    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926803    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926806    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926809    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926812    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926814    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926816    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926819    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926821    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926824    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926826    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926831    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926833    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926836    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926838    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926841    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926843    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926846    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926848    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:48:38.928963 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926850    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926853    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926861    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926864    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926867    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926871    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926875    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926877    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926880    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926883    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926886    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926889    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926892    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926895    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926898    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926900    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926903    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926907    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:48:38.929455 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926910    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926913    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926916    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926918    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926921    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926924    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926928    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926931    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926934    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926936    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926939    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926941    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926944    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926947    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926949    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926952    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926954    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926964    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926966    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:48:38.929959 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926969    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926977    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926980    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926982    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926985    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926987    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926990    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926992    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926995    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.926998    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.927000    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.927003    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.927005    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.927007    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928327    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928336    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928339    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928343    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928345    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928350    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:48:38.930423 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928353    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928356    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928359    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928363    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928365    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928368    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928371    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928374    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928376    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928379    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928388    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928392    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928394    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928397    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928399    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928402    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928412    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928414    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928417    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928420    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:48:38.930936 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928422    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928426    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928429    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928431    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928434    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928436    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928439    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928442    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928444    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928446    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928449    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928452    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928454    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928457    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928460    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928462    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928465    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928467    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928470    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:48:38.931432 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928473    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928476    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928478    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928481    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928490    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928493    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928495    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928498    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928500    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928503    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928508    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928511    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928513    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928516    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928518    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928523    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928526    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928529    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928532    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928534    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:48:38.931944 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928537    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928539    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928542    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928546    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928549    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928553    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928555    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928558    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928562    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928584    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928589    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928593    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928596    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928598    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928602    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928604    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928607    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928615    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928618    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:48:38.932545 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928621    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928641    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928644    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928647    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928649    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928652    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928654    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928657    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928660    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928662    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928665    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928667    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928670    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928673    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.928675    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928759    2563 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928813    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928824    2563 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928829    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928833    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928836    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 13:48:38.933032 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928842    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928847    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928850    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928853    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928856    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928860    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928863    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928866    2563 flags.go:64] FLAG: --cgroup-root=""
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928869    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928872    2563 flags.go:64] FLAG: --client-ca-file=""
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928882    2563 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928885    2563 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928893    2563 flags.go:64] FLAG: --cluster-domain=""
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928895    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928898    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928902    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928905    2563 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928909    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928912    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928915    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928918    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928921    2563 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928924    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928927    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 13:48:38.933538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928930    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928933    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928937    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928940    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928945    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928948    2563 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928951    2563 flags.go:64] FLAG: --enable-server="true"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928954    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928963    2563 flags.go:64] FLAG: --event-burst="100"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928966    2563 flags.go:64] FLAG: --event-qps="50"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928969    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928972    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928975    2563 flags.go:64] FLAG: --eviction-hard=""
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928979    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928983    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928986    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928988    2563 flags.go:64] FLAG: --eviction-soft=""
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928991    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.928994    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929003    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929006    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929009    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929012    2563 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929014    2563 flags.go:64] FLAG: --feature-gates=""
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929018    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 13:48:38.934144 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929021    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929025    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929028    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929030    2563 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929033    2563 flags.go:64] FLAG: --help="false"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929036    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-128-161.ec2.internal"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929040    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929042    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929045    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929049    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929052    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929056    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929059    2563 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929062    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929065    2563 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929068    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929072    2563 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929074    2563 flags.go:64] FLAG: --kube-reserved=""
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929077    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929080    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929084    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929086    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929089    2563 flags.go:64] FLAG: --lock-file=""
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929092    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 13:48:38.934848 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929095    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929098    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929103    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929114    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929118    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929120    2563 flags.go:64] FLAG: --logging-format="text"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929123    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929126    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929129    2563 flags.go:64] FLAG: --manifest-url=""
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929132    2563 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929137    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929140    2563 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929143    2563 flags.go:64] FLAG: --max-pods="110"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929146    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929149    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929152    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929155    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929158    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929161    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929166    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929175    2563 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929178    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929181    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 13:48:38.935458 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929184    2563 flags.go:64] FLAG: --pod-cidr=""
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929187    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929192    2563 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929195    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929198    2563 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929201    2563 flags.go:64] FLAG: --port="10250"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929204    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929207    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d24d9416d90aa6c7"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929210    2563 flags.go:64] FLAG: --qos-reserved=""
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929213    2563 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929216    2563 flags.go:64] FLAG: --register-node="true"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929219    2563 flags.go:64] FLAG: --register-with-taints=""
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929223    2563 flags.go:64] FLAG: --registry-burst="10"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929230    2563 flags.go:64] FLAG: --registry-qps="5"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929232    2563 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929235    2563 flags.go:64] FLAG: --reserved-memory=""
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929239    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929242    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929245    2563 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929248    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929251    2563 flags.go:64] FLAG: --runonce="false"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929254    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929257    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929260    2563 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929263    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 13:48:38.936049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929266    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929269    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929272    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929275    2563 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929279    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929282    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929286    2563 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929289    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929292    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929294    2563 flags.go:64] FLAG: --system-cgroups=""
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929297    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929302    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929305    2563 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929308    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929313    2563 flags.go:64] FLAG: --tls-min-version=""
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929315    2563 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929318    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929321    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929324    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929327    2563 flags.go:64] FLAG: --v="2"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929331    2563 flags.go:64] FLAG: --version="false"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929336    2563 flags.go:64] FLAG: --vmodule=""
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929340    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.929343    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929483    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:48:38.936700 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929486    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929490    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929493    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929495    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929498    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929501    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929504    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929506    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929510    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929512    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929515    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929519    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929521    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929524    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929527    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929529    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929532    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929534    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929537    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929540    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:48:38.937343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929542    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929545    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929547    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929550    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929552    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929555    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929557    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929560    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929562    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929565    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929568    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929571    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929573    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929576    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929578    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929581    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929583    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929586    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929588    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929591    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:48:38.937883 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929593    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929595    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929598    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929603    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929606    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929608    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929612    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929615    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929618    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929620    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929638    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929641    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929644    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929646    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929649    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929652    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929654    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929657    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929659    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929662    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:48:38.938380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929665    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929667    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929671    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929673    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929676    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929678    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929681    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929684    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929688    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929692    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929694    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929697    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929699    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929702    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929705    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929710    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929712    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929715    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929719    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:48:38.938990 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929722    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929724    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929728    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929732    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929735    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929739    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929741    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929745    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929748    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929751    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929754    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929756    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929759    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929763    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929765    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929768    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929771    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929774    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:48:38.939468 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.929777    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:48:38.939955 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.930537    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:48:38.939955 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.939302    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 13:48:38.939955 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.939327    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 13:48:38.939955 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.939354    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 13:48:38.939955 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.939361    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 13:48:38.939955 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939446    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:48:38.939955 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939453    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:48:38.939955 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939456    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:48:38.939955 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939459    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:48:38.939955 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939461    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:48:38.939955 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939465    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:48:38.939955 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939468    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:48:38.939955 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939471    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:48:38.939955 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939474    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:48:38.939955 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939477    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939479    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939482    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939485    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939489    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939494    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939498    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939502    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939505    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939508    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939512    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939516    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939519    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939522    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939525    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939527    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939530    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939532    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939535    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939537    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:48:38.940331 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939543    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939546    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939549    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939551    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939556    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939560    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939563    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939567    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939571    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939576    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939580    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939583    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939586    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939588    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939590    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939593    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939596    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939598    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939601    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:48:38.940841 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939603    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939606    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939609    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939612    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939614    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939616    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939619    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939638    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939644    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939648    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939651    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939654    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939657    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939661    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939666    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939669    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939671    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939674    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939677    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:48:38.941311 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939680    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939683    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939685    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939688    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939690    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939692    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939695    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939698    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939700    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939703    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939705    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939708    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939711    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939715    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939720    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939724    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939728    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939731    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939733    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939736    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:48:38.941805 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939738    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:48:38.942296 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939743    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:48:38.942296 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939746    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:48:38.942296 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939750    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:48:38.942296 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939753    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:48:38.942296 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939756    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:48:38.942296 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939759    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:48:38.942296 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939762    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:48:38.942296 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939766    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:48:38.942296 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939769    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:48:38.942296 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939772    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:48:38.942296 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939775    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:48:38.942296 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.939781    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:48:38.942296 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939953    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:48:38.942296 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939960    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939965    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939970    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939972    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939975    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939978    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939982    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939984    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939987    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939990    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939992    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939995    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.939998    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940000    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940003    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940006    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940009    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940011    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940014    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:48:38.942802 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940017    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940019    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940022    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940025    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940029    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940034    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940038    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940041    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940044    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940047    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940049    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940051    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940054    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940056    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940059    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940062    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940064    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940067    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940069    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940072    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:48:38.943343 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940075    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940077    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940080    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940084    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940087    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940089    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940092    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940095    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940097    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940101    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940105    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940110    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940114    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940117    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940120    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940123    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940125    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940128    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940131    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940133    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:48:38.943856 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940136    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940140    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940142    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940145    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940148    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940150    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940153    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940155    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940158    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940160    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940163    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940166    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940169    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940171    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940173    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940176    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940180    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940184    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940191    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:48:38.944419 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940196    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940200    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940202    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940205    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940207    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940210    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940212    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940215    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940217    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940220    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940222    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940228    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940231    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940233    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940236    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940239    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940242    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940245    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940247    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:48:38.944902 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:38.940250    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:48:38.945367 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.940255    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:48:38.945367 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.940441    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 13:48:38.945367 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.942961    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 13:48:38.945367 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.944032    2563 server.go:1019] "Starting client certificate rotation"
Sep 11 13:48:38.945367 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.944152    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:48:38.945367 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.944198    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:48:38.946989 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.946969    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:48:38.949468 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.949415    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:48:38.961051 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.961025    2563 log.go:25] "Validated CRI v1 runtime API"
Sep 11 13:48:38.966050 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.966021    2563 log.go:25] "Validated CRI v1 image API"
Sep 11 13:48:38.967091 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.967073    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 13:48:38.969892 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.969865    2563 fs.go:135] Filesystem UUIDs: map[567ba786-072c-4fd7-879b-49e0d0448163:/dev/nvme0n1p3 605dbdcc-5dee-4f3a-bf70-923d17e2bf59:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 11 13:48:38.969979 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.969889    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 13:48:38.974730 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.974696    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:48:38.975285 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.975156    2563 manager.go:216] Machine: {Timestamp:2026-09-11 13:48:38.973685412 +0000 UTC m=+0.371028167 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3094872 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25937134d8afc28dfec204a5e2193a SystemUUID:ec259371-34d8-afc2-8dfe-c204a5e2193a BootID:f3fe2fb5-07fa-4808-82f4-ee0ee599659e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:08:75:84:45:4d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:08:75:84:45:4d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:4f:cd:73:5a:b1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 13:48:38.975393 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.975290    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 13:48:38.975492 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.975474    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 13:48:38.979242 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.979195    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 13:48:38.979440 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.979243    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-161.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 13:48:38.980245 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.980228    2563 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 13:48:38.980301 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.980252    2563 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 13:48:38.980301 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.980274    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:48:38.981182 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.981167    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:48:38.982371 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.982356    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 13:48:38.985952 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.985931    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:48:38.986123 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.986105    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 13:48:38.986199 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.986152    2563 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 13:48:38.986199 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.986171    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 13:48:38.986199 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.986194    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 13:48:38.986315 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.986208    2563 kubelet.go:398] "Adding apiserver pod source"
Sep 11 13:48:38.986315 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.986231    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 13:48:38.986730 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.986703    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:48:38.986802 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.986742    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:48:38.989403 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.989383    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 13:48:38.992091 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.992072    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 13:48:38.992170 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.992105    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 13:48:38.994257 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.994242    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 13:48:38.994326 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.994260    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 13:48:38.994326 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.994267    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 13:48:38.994326 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.994274    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 13:48:38.994326 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.994285    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 13:48:38.994326 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.994294    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 13:48:38.994326 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.994310    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 13:48:38.994326 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.994316    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 13:48:38.994326 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.994322    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 13:48:38.994326 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.994329    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 13:48:38.994565 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.994339    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 13:48:38.994565 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.994344    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 13:48:38.995232 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.995221    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 13:48:38.995263 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.995233    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 13:48:38.997413 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.997394    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-161.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 13:48:38.997470 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:38.997439    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 13:48:38.997470 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:38.997439    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-161.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 13:48:38.999456 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.999442    2563 server.go:1268] "Started kubelet"
Sep 11 13:48:38.999602 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.999537    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 13:48:38.999736 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.999640    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 13:48:38.999797 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:38.999672    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 13:48:39.000405 ip-10-0-128-161 systemd[1]: Started Kubernetes Kubelet.
Sep 11 13:48:39.002181 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.002159    2563 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 13:48:39.002278 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.002272    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 13:48:39.003597 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.003575    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cpk9n"
Sep 11 13:48:39.005727 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.005708    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 13:48:39.007864 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.007845    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 13:48:39.007939 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.007858    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 13:48:39.008004 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.007954    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:39.008695 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.008675    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 13:48:39.009195 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.008999    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 13:48:39.009281 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.009200    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 13:48:39.009558 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.009540    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 13:48:39.010003 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.009559    2563 factory.go:55] Registering systemd factory
Sep 11 13:48:39.010003 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.009568    2563 factory.go:223] Registration of the systemd container factory successfully
Sep 11 13:48:39.010314 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.010006    2563 factory.go:153] Registering CRI-O factory
Sep 11 13:48:39.010314 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.010066    2563 factory.go:223] Registration of the crio container factory successfully
Sep 11 13:48:39.010314 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.010086    2563 factory.go:103] Registering Raw factory
Sep 11 13:48:39.010314 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.010116    2563 manager.go:1196] Started watching for new ooms in manager
Sep 11 13:48:39.010863 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.010320    2563 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 13:48:39.010863 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.010331    2563 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 13:48:39.011829 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.011802    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cpk9n"
Sep 11 13:48:39.014104 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.014084    2563 manager.go:349] Starting recovery of all containers
Sep 11 13:48:39.015714 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.015683    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 13:48:39.015838 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.015782    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-161.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 13:48:39.017023 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.015754    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-161.ec2.internal.18d4485da4595bda  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-161.ec2.internal,UID:ip-10-0-128-161.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-161.ec2.internal,},FirstTimestamp:2026-09-11 13:48:38.999407578 +0000 UTC m=+0.396750306,LastTimestamp:2026-09-11 13:48:38.999407578 +0000 UTC m=+0.396750306,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-161.ec2.internal,}"
Sep 11 13:48:39.025300 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.025282    2563 manager.go:354] Recovery completed
Sep 11 13:48:39.029533 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.029512    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:48:39.031524 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.031502    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:48:39.031606 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.031543    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:48:39.031606 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.031555    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:48:39.032251 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.032226    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 13:48:39.032251 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.032240    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 13:48:39.032376 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.032260    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:48:39.032376 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.032233    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:48:39.033109 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.033092    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:48:39.033192 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.033119    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:48:39.033192 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.033129    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:48:39.033415 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.033404    2563 policy_none.go:49] "None policy: Start"
Sep 11 13:48:39.033452 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.033420    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 13:48:39.033452 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.033432    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 13:48:39.034254 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.034243    2563 policy_none.go:47] "Start"
Sep 11 13:48:39.085399 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.085376    2563 manager.go:337] "Starting Device Plugin manager"
Sep 11 13:48:39.095406 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.085422    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 13:48:39.095406 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.085432    2563 server.go:87] "Starting device plugin registration server"
Sep 11 13:48:39.095406 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.085719    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 13:48:39.095406 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.085729    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 13:48:39.095406 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.085863    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 13:48:39.095406 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.085933    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 13:48:39.095406 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.085940    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 13:48:39.095406 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.087220    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 13:48:39.095406 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.087283    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:39.123081 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.123035    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 13:48:39.124450 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.124428    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 13:48:39.124571 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.124483    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 13:48:39.124571 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.124510    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 13:48:39.124571 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.124551    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 13:48:39.127576 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.127554    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:48:39.186859 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.186770    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:48:39.187844 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.187825    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:48:39.187965 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.187864    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:48:39.187965 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.187881    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:48:39.187965 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.187925    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.196457 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.196426    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.196544 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.196460    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-161.ec2.internal\": node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:39.216839 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.216804    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:39.225203 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.225173    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal"]
Sep 11 13:48:39.225284 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.225251    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:48:39.226859 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.226838    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:48:39.226993 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.226876    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:48:39.226993 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.226890    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:48:39.229224 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.229209    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:48:39.229344 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.229324    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.229398 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.229360    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:48:39.230170 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.230148    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:48:39.230266 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.230186    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:48:39.230266 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.230198    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:48:39.230266 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.230149    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:48:39.230266 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.230236    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:48:39.230266 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.230247    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:48:39.232444 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.232427    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.232493 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.232457    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:48:39.233357 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.233338    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:48:39.233433 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.233380    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:48:39.233433 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.233395    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:48:39.257640 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.257604    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-161.ec2.internal\" not found" node="ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.263935 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.263908    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-161.ec2.internal\" not found" node="ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.312100 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.312070    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fbf860b5b0097ab5f709024d70f9c3d5-config\") pod \"kube-apiserver-proxy-ip-10-0-128-161.ec2.internal\" (UID: \"fbf860b5b0097ab5f709024d70f9c3d5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.312229 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.312117    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e80b9e5ab597f3b17c8733c311f97f1f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal\" (UID: \"e80b9e5ab597f3b17c8733c311f97f1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.312229 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.312169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e80b9e5ab597f3b17c8733c311f97f1f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal\" (UID: \"e80b9e5ab597f3b17c8733c311f97f1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.317232 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.317203    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:39.413298 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.413254    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e80b9e5ab597f3b17c8733c311f97f1f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal\" (UID: \"e80b9e5ab597f3b17c8733c311f97f1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.413298 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.413281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e80b9e5ab597f3b17c8733c311f97f1f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal\" (UID: \"e80b9e5ab597f3b17c8733c311f97f1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.413532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.413333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e80b9e5ab597f3b17c8733c311f97f1f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal\" (UID: \"e80b9e5ab597f3b17c8733c311f97f1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.413532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.413354    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fbf860b5b0097ab5f709024d70f9c3d5-config\") pod \"kube-apiserver-proxy-ip-10-0-128-161.ec2.internal\" (UID: \"fbf860b5b0097ab5f709024d70f9c3d5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.413532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.413391    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fbf860b5b0097ab5f709024d70f9c3d5-config\") pod \"kube-apiserver-proxy-ip-10-0-128-161.ec2.internal\" (UID: \"fbf860b5b0097ab5f709024d70f9c3d5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.413532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.413427    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e80b9e5ab597f3b17c8733c311f97f1f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal\" (UID: \"e80b9e5ab597f3b17c8733c311f97f1f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.417318 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.417272    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:39.518218 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.518149    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:39.559367 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.559325    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.567370 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.567346    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal"
Sep 11 13:48:39.619267 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.619214    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:39.719881 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.719841    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:39.820448 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.820357    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:39.921193 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:39.921024    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:39.944239 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.944196    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 13:48:39.944432 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:39.944409    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 13:48:40.017788 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.017739    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 13:43:39 +0000 UTC" deadline="2028-06-15 08:54:42.125578152 +0000 UTC"
Sep 11 13:48:40.017788 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.017775    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15427h6m2.107807764s"
Sep 11 13:48:40.021891 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:40.021868    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:40.084084 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.083984    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:48:40.119684 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:40.119621    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbf860b5b0097ab5f709024d70f9c3d5.slice/crio-9251d3cf847c7a6b22d40a3e993b1311a21525919661292b5f0b7b9e9786befd WatchSource:0}: Error finding container 9251d3cf847c7a6b22d40a3e993b1311a21525919661292b5f0b7b9e9786befd: Status 404 returned error can't find the container with id 9251d3cf847c7a6b22d40a3e993b1311a21525919661292b5f0b7b9e9786befd
Sep 11 13:48:40.119976 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:40.119953    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode80b9e5ab597f3b17c8733c311f97f1f.slice/crio-2ad6db46c75c51e9ebbc444adcdf7a2a968bbbf551770ae0e53579811d9138d0 WatchSource:0}: Error finding container 2ad6db46c75c51e9ebbc444adcdf7a2a968bbbf551770ae0e53579811d9138d0: Status 404 returned error can't find the container with id 2ad6db46c75c51e9ebbc444adcdf7a2a968bbbf551770ae0e53579811d9138d0
Sep 11 13:48:40.122607 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:40.122287    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:40.126572 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.126551    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:48:40.128053 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.128004    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" event={"ID":"e80b9e5ab597f3b17c8733c311f97f1f","Type":"ContainerStarted","Data":"2ad6db46c75c51e9ebbc444adcdf7a2a968bbbf551770ae0e53579811d9138d0"}
Sep 11 13:48:40.129314 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.129289    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal" event={"ID":"fbf860b5b0097ab5f709024d70f9c3d5","Type":"ContainerStarted","Data":"9251d3cf847c7a6b22d40a3e993b1311a21525919661292b5f0b7b9e9786befd"}
Sep 11 13:48:40.222968 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:40.222922    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:40.323827 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:40.323786    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:40.370464 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.370377    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:48:40.424381 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:40.424340    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:40.524705 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:40.524658    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found"
Sep 11 13:48:40.524705 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.524692    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:48:40.608509 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.608473    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal"
Sep 11 13:48:40.619030 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.618987    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:48:40.620194 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.620164    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal"
Sep 11 13:48:40.630544 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.630454    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:48:40.909608 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.909518    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:48:40.987148 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.987099    2563 apiserver.go:52] "Watching apiserver"
Sep 11 13:48:40.993650 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.993612    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 13:48:40.996274 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.996232    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-kdcfl","kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal","openshift-cluster-node-tuning-operator/tuned-p7vp4","openshift-image-registry/node-ca-5frp2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal","openshift-multus/multus-additional-cni-plugins-jkd4s","openshift-multus/network-metrics-daemon-59892","openshift-network-operator/iptables-alerter-bc9vh","kube-system/global-pull-secret-syncer-hfjt6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq","openshift-dns/node-resolver-7q9dt","openshift-multus/multus-kwvws","openshift-network-diagnostics/network-check-target-lrb9b","openshift-ovn-kubernetes/ovnkube-node-c4hqn"]
Sep 11 13:48:40.999274 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:40.999241    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.001955 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.001930    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 13:48:41.002281 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.002259    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 13:48:41.002511 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.002494    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 13:48:41.004570 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.004536    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7q9dt"
Sep 11 13:48:41.004716 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.004622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.008174 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.005367    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-78m7q\""
Sep 11 13:48:41.008368 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.008231    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bc9vh"
Sep 11 13:48:41.008726 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.008536    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 13:48:41.008726 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.008623    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 13:48:41.009262 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.008983    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nrtzh\""
Sep 11 13:48:41.009262 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.009054    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 13:48:41.009262 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.009201    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hfrnc\""
Sep 11 13:48:41.009524 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.009280    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 13:48:41.009524 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.009338    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 13:48:41.009524 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.009405    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 13:48:41.009524 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.009520    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 13:48:41.009872 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.009632    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 13:48:41.009872 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.009747    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 13:48:41.013169 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.010936    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.013169 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.012523    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-x4c9z\""
Sep 11 13:48:41.013169 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.012982    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:48:41.013762 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.013735    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 13:48:41.013858 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.013825    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 13:48:41.013918 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.013892    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 13:48:41.014065 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.013811    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 13:48:41.014148 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.014105    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 13:48:41.014431 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.014410    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hrtrm\""
Sep 11 13:48:41.014538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.014489    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 13:48:41.018162 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.018137    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:41.018294 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.018225    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:48:41.019056 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.019033    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:48:41.021214 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.021192    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kdcfl"
Sep 11 13:48:41.023514 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.023462    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-schgd\""
Sep 11 13:48:41.023647 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.023546    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 13:48:41.023774 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.023758    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 13:48:41.024067 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-registration-dir\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.024143 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4ll9\" (UniqueName: \"kubernetes.io/projected/e4d48da9-3103-4bba-8652-483670f758b4-kube-api-access-k4ll9\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.024143 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-slash\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.024243 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024155    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-run-netns\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.024297 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024252    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrj4b\" (UniqueName: \"kubernetes.io/projected/1175fe53-1a2b-424b-a493-08c8d6ec865b-kube-api-access-hrj4b\") pod \"iptables-alerter-bc9vh\" (UID: \"1175fe53-1a2b-424b-a493-08c8d6ec865b\") " pod="openshift-network-operator/iptables-alerter-bc9vh"
Sep 11 13:48:41.024349 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024296    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-system-cni-dir\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.024349 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024325    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-var-lib-cni-bin\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.024467 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-multus-daemon-config\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.024467 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-kubelet\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.024467 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024454    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.024624 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1675e538-4e61-4929-8935-2e31abb0299a-ovnkube-config\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.024624 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024508    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-run-systemd\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.024624 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-run-ovn-kubernetes\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.024624 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024570    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1675e538-4e61-4929-8935-2e31abb0299a-ovnkube-script-lib\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.024624 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-run-netns\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.024893 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024702    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-var-lib-cni-multus\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.024893 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024752    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-var-lib-kubelet\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.024893 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-node-log\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.024893 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024832    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-cni-bin\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.024893 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024859    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1175fe53-1a2b-424b-a493-08c8d6ec865b-host-slash\") pod \"iptables-alerter-bc9vh\" (UID: \"1175fe53-1a2b-424b-a493-08c8d6ec865b\") " pod="openshift-network-operator/iptables-alerter-bc9vh"
Sep 11 13:48:41.024893 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024885    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-cni-binary-copy\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.025117 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024910    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-socket-dir\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.025117 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-device-dir\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.025117 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.024979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8hwc\" (UniqueName: \"kubernetes.io/projected/1675e538-4e61-4929-8935-2e31abb0299a-kube-api-access-v8hwc\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.025117 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025003    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-hostroot\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.025117 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-run-multus-certs\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.025117 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-etc-kubernetes\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.025117 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025106    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lmz7\" (UniqueName: \"kubernetes.io/projected/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-kube-api-access-5lmz7\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.025429 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071b3477-ebf8-450b-9ca7-c2767dea40ea-tmp-dir\") pod \"node-resolver-7q9dt\" (UID: \"071b3477-ebf8-450b-9ca7-c2767dea40ea\") " pod="openshift-dns/node-resolver-7q9dt"
Sep 11 13:48:41.025429 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxh55\" (UniqueName: \"kubernetes.io/projected/071b3477-ebf8-450b-9ca7-c2767dea40ea-kube-api-access-qxh55\") pod \"node-resolver-7q9dt\" (UID: \"071b3477-ebf8-450b-9ca7-c2767dea40ea\") " pod="openshift-dns/node-resolver-7q9dt"
Sep 11 13:48:41.025429 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025178    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-etc-openvswitch\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.025429 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-cnibin\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.025429 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025232    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-sys-fs\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.025429 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025287    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-systemd-units\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.025429 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025323    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-run-openvswitch\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.025429 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025355    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-run-ovn\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.025429 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025385    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-log-socket\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.025429 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-cni-netd\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.025846 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025456    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-multus-cni-dir\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.025846 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025481    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-multus-socket-dir-parent\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.025846 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025506    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.025846 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025557    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-etc-selinux\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.025846 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025578    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/071b3477-ebf8-450b-9ca7-c2767dea40ea-hosts-file\") pod \"node-resolver-7q9dt\" (UID: \"071b3477-ebf8-450b-9ca7-c2767dea40ea\") " pod="openshift-dns/node-resolver-7q9dt"
Sep 11 13:48:41.025846 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025604    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-var-lib-openvswitch\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.025846 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1675e538-4e61-4929-8935-2e31abb0299a-env-overrides\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.025846 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025656    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1675e538-4e61-4929-8935-2e31abb0299a-ovn-node-metrics-cert\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.025846 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025683    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1175fe53-1a2b-424b-a493-08c8d6ec865b-iptables-alerter-script\") pod \"iptables-alerter-bc9vh\" (UID: \"1175fe53-1a2b-424b-a493-08c8d6ec865b\") " pod="openshift-network-operator/iptables-alerter-bc9vh"
Sep 11 13:48:41.025846 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025716    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-run-k8s-cni-cncf-io\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.025846 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.025751    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-multus-conf-dir\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.026944 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.026925    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:41.027039 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.026998    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:48:41.030744 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.030681    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:41.030848 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.030827    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:48:41.033476 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.033455    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.036029 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.035773    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 13:48:41.036029 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.035825    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 13:48:41.036029 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.035824    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9lplj\""
Sep 11 13:48:41.036748 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.036674    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5frp2"
Sep 11 13:48:41.038900 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.038878    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 13:48:41.039021 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.038914    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 13:48:41.039021 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.038948    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 13:48:41.039241 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.039219    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-df6pg\""
Sep 11 13:48:41.039675 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.039656    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.041934 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.041884    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ppkxx\""
Sep 11 13:48:41.042065 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.041964    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:48:41.042135 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.042100    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 13:48:41.045351 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.045325    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8qghm"
Sep 11 13:48:41.053728 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.053675    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8qghm"
Sep 11 13:48:41.111059 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.111017    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 13:48:41.126149 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126110    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-log-socket\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.126149 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-cni-netd\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.126398 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126192    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1675e538-4e61-4929-8935-2e31abb0299a-env-overrides\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.126398 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126228    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb93b48e-5389-4693-af4a-52e1e2163832-agent-certs\") pod \"konnectivity-agent-kdcfl\" (UID: \"cb93b48e-5389-4693-af4a-52e1e2163832\") " pod="kube-system/konnectivity-agent-kdcfl"
Sep 11 13:48:41.126398 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126263    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-multus-socket-dir-parent\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.126398 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126289    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.126398 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126321    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-etc-selinux\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.126398 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126353    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-var-lib-openvswitch\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.126398 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1675e538-4e61-4929-8935-2e31abb0299a-ovn-node-metrics-cert\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.126725 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85429\" (UniqueName: \"kubernetes.io/projected/3896a84c-a6ea-468d-8e83-cded2f536e6d-kube-api-access-85429\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.126725 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126446    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1175fe53-1a2b-424b-a493-08c8d6ec865b-iptables-alerter-script\") pod \"iptables-alerter-bc9vh\" (UID: \"1175fe53-1a2b-424b-a493-08c8d6ec865b\") " pod="openshift-network-operator/iptables-alerter-bc9vh"
Sep 11 13:48:41.126725 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-run-k8s-cni-cncf-io\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.126725 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126510    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-registration-dir\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.126725 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-slash\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.126725 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126577    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb93b48e-5389-4693-af4a-52e1e2163832-konnectivity-ca\") pod \"konnectivity-agent-kdcfl\" (UID: \"cb93b48e-5389-4693-af4a-52e1e2163832\") " pod="kube-system/konnectivity-agent-kdcfl"
Sep 11 13:48:41.126725 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126627    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-sysctl-d\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.126725 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126658    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-run\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.126725 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126691    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/137f87ff-936c-417c-88f9-33c2a48d721f-cnibin\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.127064 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126760    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-system-cni-dir\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.127064 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126796    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-kubelet\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.127064 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.127064 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1675e538-4e61-4929-8935-2e31abb0299a-ovnkube-config\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.127064 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126914    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-kubernetes\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.127064 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126947    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/137f87ff-936c-417c-88f9-33c2a48d721f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.127064 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.126973    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2c293f3c-09e5-4f86-991b-1613158f19ee-kubelet-config\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:41.127064 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127007    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-node-log\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.127064 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127037    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-cni-bin\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.127390 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127069    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-modprobe-d\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.127390 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-hostroot\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.127390 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127124    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-etc-kubernetes\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.127390 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071b3477-ebf8-450b-9ca7-c2767dea40ea-tmp-dir\") pod \"node-resolver-7q9dt\" (UID: \"071b3477-ebf8-450b-9ca7-c2767dea40ea\") " pod="openshift-dns/node-resolver-7q9dt"
Sep 11 13:48:41.127390 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxh55\" (UniqueName: \"kubernetes.io/projected/071b3477-ebf8-450b-9ca7-c2767dea40ea-kube-api-access-qxh55\") pod \"node-resolver-7q9dt\" (UID: \"071b3477-ebf8-450b-9ca7-c2767dea40ea\") " pod="openshift-dns/node-resolver-7q9dt"
Sep 11 13:48:41.127390 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127217    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-etc-openvswitch\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.127390 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127250    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-run-ovn\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.127390 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127290    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d39e8d10-0ae8-4265-9b99-ac2ac072b1ab-host\") pod \"node-ca-5frp2\" (UID: \"d39e8d10-0ae8-4265-9b99-ac2ac072b1ab\") " pod="openshift-image-registry/node-ca-5frp2"
Sep 11 13:48:41.127390 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127317    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68kcp\" (UniqueName: \"kubernetes.io/projected/d39e8d10-0ae8-4265-9b99-ac2ac072b1ab-kube-api-access-68kcp\") pod \"node-ca-5frp2\" (UID: \"d39e8d10-0ae8-4265-9b99-ac2ac072b1ab\") " pod="openshift-image-registry/node-ca-5frp2"
Sep 11 13:48:41.127390 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-cnibin\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.127390 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-systemd-units\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.127827 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127411    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-run-openvswitch\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.127827 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127443    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x97z\" (UniqueName: \"kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z\") pod \"network-check-target-lrb9b\" (UID: \"98c33f0c-094f-490f-af00-24b762506101\") " pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:41.127827 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-sysconfig\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.127827 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127497    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-sys\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.127827 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-host\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.127827 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127557    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs\") pod \"network-metrics-daemon-59892\" (UID: \"47db31ca-5cef-4c37-9da5-027f3df08815\") " pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:41.127827 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-multus-cni-dir\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.127827 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127616    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/071b3477-ebf8-450b-9ca7-c2767dea40ea-hosts-file\") pod \"node-resolver-7q9dt\" (UID: \"071b3477-ebf8-450b-9ca7-c2767dea40ea\") " pod="openshift-dns/node-resolver-7q9dt"
Sep 11 13:48:41.127827 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127650    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmv5r\" (UniqueName: \"kubernetes.io/projected/137f87ff-936c-417c-88f9-33c2a48d721f-kube-api-access-kmv5r\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.127827 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127685    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-multus-conf-dir\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.127827 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127740    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4ll9\" (UniqueName: \"kubernetes.io/projected/e4d48da9-3103-4bba-8652-483670f758b4-kube-api-access-k4ll9\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.127827 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-run-netns\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.127827 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127803    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-tuned\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127865    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/137f87ff-936c-417c-88f9-33c2a48d721f-system-cni-dir\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127896    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/137f87ff-936c-417c-88f9-33c2a48d721f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127925    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrj4b\" (UniqueName: \"kubernetes.io/projected/1175fe53-1a2b-424b-a493-08c8d6ec865b-kube-api-access-hrj4b\") pod \"iptables-alerter-bc9vh\" (UID: \"1175fe53-1a2b-424b-a493-08c8d6ec865b\") " pod="openshift-network-operator/iptables-alerter-bc9vh"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-var-lib-cni-bin\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.127990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-multus-daemon-config\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1675e538-4e61-4929-8935-2e31abb0299a-ovnkube-script-lib\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128054    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d39e8d10-0ae8-4265-9b99-ac2ac072b1ab-serviceca\") pod \"node-ca-5frp2\" (UID: \"d39e8d10-0ae8-4265-9b99-ac2ac072b1ab\") " pod="openshift-image-registry/node-ca-5frp2"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-sysctl-conf\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128115    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-lib-modules\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128147    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7t6h\" (UniqueName: \"kubernetes.io/projected/47db31ca-5cef-4c37-9da5-027f3df08815-kube-api-access-d7t6h\") pod \"network-metrics-daemon-59892\" (UID: \"47db31ca-5cef-4c37-9da5-027f3df08815\") " pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128180    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-run-systemd\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128212    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-run-ovn-kubernetes\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-systemd\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3896a84c-a6ea-468d-8e83-cded2f536e6d-tmp\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128302    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/137f87ff-936c-417c-88f9-33c2a48d721f-cni-binary-copy\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.128407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128342    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-run-netns\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.129025 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-var-lib-cni-multus\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.129025 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128427    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-var-lib-kubelet\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.129025 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128460    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2c293f3c-09e5-4f86-991b-1613158f19ee-dbus\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:41.129025 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128493    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1175fe53-1a2b-424b-a493-08c8d6ec865b-host-slash\") pod \"iptables-alerter-bc9vh\" (UID: \"1175fe53-1a2b-424b-a493-08c8d6ec865b\") " pod="openshift-network-operator/iptables-alerter-bc9vh"
Sep 11 13:48:41.129025 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128523    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-cni-binary-copy\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.129025 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-socket-dir\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.129025 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128590    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-device-dir\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.129025 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128622    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8hwc\" (UniqueName: \"kubernetes.io/projected/1675e538-4e61-4929-8935-2e31abb0299a-kube-api-access-v8hwc\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.129025 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128655    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-var-lib-kubelet\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.129025 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128687    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-run-multus-certs\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.129025 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lmz7\" (UniqueName: \"kubernetes.io/projected/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-kube-api-access-5lmz7\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.129025 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128776    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/137f87ff-936c-417c-88f9-33c2a48d721f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.129025 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128814    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-sys-fs\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.129025 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.128962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-sys-fs\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.129524 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.129032    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-log-socket\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.129524 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.129085    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-cni-netd\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.129781 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.129754    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1675e538-4e61-4929-8935-2e31abb0299a-env-overrides\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.130005 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.129983    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-multus-socket-dir-parent\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.130102 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.130061    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.130210 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.130159    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-etc-selinux\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.130259 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.130210    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-var-lib-openvswitch\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.130600 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.130573    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 13:48:41.133167 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.133138    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-multus-cni-dir\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.133941 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.133836    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1175fe53-1a2b-424b-a493-08c8d6ec865b-iptables-alerter-script\") pod \"iptables-alerter-bc9vh\" (UID: \"1175fe53-1a2b-424b-a493-08c8d6ec865b\") " pod="openshift-network-operator/iptables-alerter-bc9vh"
Sep 11 13:48:41.134455 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.134430    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-run-k8s-cni-cncf-io\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.134547 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.134527    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-registration-dir\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.134610 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.134600    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-slash\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.134668 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.134643    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-multus-conf-dir\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.134668 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.134659    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/071b3477-ebf8-450b-9ca7-c2767dea40ea-hosts-file\") pod \"node-resolver-7q9dt\" (UID: \"071b3477-ebf8-450b-9ca7-c2767dea40ea\") " pod="openshift-dns/node-resolver-7q9dt"
Sep 11 13:48:41.134834 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.134815    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-system-cni-dir\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.134893 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.134881    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-kubelet\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.134949 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.134934    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.135072 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.135051    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-run-netns\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.135508 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.135456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-var-lib-cni-bin\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.135508 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.135491    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1675e538-4e61-4929-8935-2e31abb0299a-ovnkube-config\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.135937 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.135916    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-etc-openvswitch\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.136026 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.135974    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-run-ovn\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.136090 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.136066    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-cnibin\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.136162 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.136104    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-systemd-units\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.136162 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.136134    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-run-openvswitch\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.136162 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.136142    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1675e538-4e61-4929-8935-2e31abb0299a-ovnkube-script-lib\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.136500    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-node-log\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.136557    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-run-systemd\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.136559    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-hostroot\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.136597    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-etc-kubernetes\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.136629    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-cni-bin\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.136639    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1175fe53-1a2b-424b-a493-08c8d6ec865b-host-slash\") pod \"iptables-alerter-bc9vh\" (UID: \"1175fe53-1a2b-424b-a493-08c8d6ec865b\") " pod="openshift-network-operator/iptables-alerter-bc9vh"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.136641    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-multus-daemon-config\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.136716    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-run-netns\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.136770    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1675e538-4e61-4929-8935-2e31abb0299a-host-run-ovn-kubernetes\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.136834    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-var-lib-cni-multus\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.136926    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071b3477-ebf8-450b-9ca7-c2767dea40ea-tmp-dir\") pod \"node-resolver-7q9dt\" (UID: \"071b3477-ebf8-450b-9ca7-c2767dea40ea\") " pod="openshift-dns/node-resolver-7q9dt"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.137841    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-var-lib-kubelet\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.138207    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-socket-dir\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.138275    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e4d48da9-3103-4bba-8652-483670f758b4-device-dir\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.138331    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-host-run-multus-certs\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.138825    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-cni-binary-copy\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.140799 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.139392    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1675e538-4e61-4929-8935-2e31abb0299a-ovn-node-metrics-cert\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.143669 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.143636    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrj4b\" (UniqueName: \"kubernetes.io/projected/1175fe53-1a2b-424b-a493-08c8d6ec865b-kube-api-access-hrj4b\") pod \"iptables-alerter-bc9vh\" (UID: \"1175fe53-1a2b-424b-a493-08c8d6ec865b\") " pod="openshift-network-operator/iptables-alerter-bc9vh"
Sep 11 13:48:41.143974 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.143795    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4ll9\" (UniqueName: \"kubernetes.io/projected/e4d48da9-3103-4bba-8652-483670f758b4-kube-api-access-k4ll9\") pod \"aws-ebs-csi-driver-node-zcvlq\" (UID: \"e4d48da9-3103-4bba-8652-483670f758b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.144863 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.144838    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxh55\" (UniqueName: \"kubernetes.io/projected/071b3477-ebf8-450b-9ca7-c2767dea40ea-kube-api-access-qxh55\") pod \"node-resolver-7q9dt\" (UID: \"071b3477-ebf8-450b-9ca7-c2767dea40ea\") " pod="openshift-dns/node-resolver-7q9dt"
Sep 11 13:48:41.150501 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.150474    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8hwc\" (UniqueName: \"kubernetes.io/projected/1675e538-4e61-4929-8935-2e31abb0299a-kube-api-access-v8hwc\") pod \"ovnkube-node-c4hqn\" (UID: \"1675e538-4e61-4929-8935-2e31abb0299a\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.151120 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.151101    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lmz7\" (UniqueName: \"kubernetes.io/projected/4a7cc659-3cac-4a00-9bb6-c90182f4c6c7-kube-api-access-5lmz7\") pod \"multus-kwvws\" (UID: \"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7\") " pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.230241 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7t6h\" (UniqueName: \"kubernetes.io/projected/47db31ca-5cef-4c37-9da5-027f3df08815-kube-api-access-d7t6h\") pod \"network-metrics-daemon-59892\" (UID: \"47db31ca-5cef-4c37-9da5-027f3df08815\") " pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:41.230241 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-systemd\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.230241 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230227    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3896a84c-a6ea-468d-8e83-cded2f536e6d-tmp\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.230539 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/137f87ff-936c-417c-88f9-33c2a48d721f-cni-binary-copy\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.230539 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2c293f3c-09e5-4f86-991b-1613158f19ee-dbus\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:41.230539 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230306    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-var-lib-kubelet\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.230539 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/137f87ff-936c-417c-88f9-33c2a48d721f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.230539 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230378    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb93b48e-5389-4693-af4a-52e1e2163832-agent-certs\") pod \"konnectivity-agent-kdcfl\" (UID: \"cb93b48e-5389-4693-af4a-52e1e2163832\") " pod="kube-system/konnectivity-agent-kdcfl"
Sep 11 13:48:41.230539 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85429\" (UniqueName: \"kubernetes.io/projected/3896a84c-a6ea-468d-8e83-cded2f536e6d-kube-api-access-85429\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.230539 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230433    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb93b48e-5389-4693-af4a-52e1e2163832-konnectivity-ca\") pod \"konnectivity-agent-kdcfl\" (UID: \"cb93b48e-5389-4693-af4a-52e1e2163832\") " pod="kube-system/konnectivity-agent-kdcfl"
Sep 11 13:48:41.230539 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230458    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-sysctl-d\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.230539 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230481    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-run\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.230539 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/137f87ff-936c-417c-88f9-33c2a48d721f-cnibin\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.230539 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-kubernetes\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/137f87ff-936c-417c-88f9-33c2a48d721f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230591    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2c293f3c-09e5-4f86-991b-1613158f19ee-kubelet-config\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230621    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-modprobe-d\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230653    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d39e8d10-0ae8-4265-9b99-ac2ac072b1ab-host\") pod \"node-ca-5frp2\" (UID: \"d39e8d10-0ae8-4265-9b99-ac2ac072b1ab\") " pod="openshift-image-registry/node-ca-5frp2"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68kcp\" (UniqueName: \"kubernetes.io/projected/d39e8d10-0ae8-4265-9b99-ac2ac072b1ab-kube-api-access-68kcp\") pod \"node-ca-5frp2\" (UID: \"d39e8d10-0ae8-4265-9b99-ac2ac072b1ab\") " pod="openshift-image-registry/node-ca-5frp2"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230726    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x97z\" (UniqueName: \"kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z\") pod \"network-check-target-lrb9b\" (UID: \"98c33f0c-094f-490f-af00-24b762506101\") " pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230750    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-sysconfig\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230776    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-sys\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-host\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230823    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs\") pod \"network-metrics-daemon-59892\" (UID: \"47db31ca-5cef-4c37-9da5-027f3df08815\") " pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmv5r\" (UniqueName: \"kubernetes.io/projected/137f87ff-936c-417c-88f9-33c2a48d721f-kube-api-access-kmv5r\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230884    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230910    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-tuned\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230934    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/137f87ff-936c-417c-88f9-33c2a48d721f-system-cni-dir\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230961    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/137f87ff-936c-417c-88f9-33c2a48d721f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.230995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d39e8d10-0ae8-4265-9b99-ac2ac072b1ab-serviceca\") pod \"node-ca-5frp2\" (UID: \"d39e8d10-0ae8-4265-9b99-ac2ac072b1ab\") " pod="openshift-image-registry/node-ca-5frp2"
Sep 11 13:48:41.231121 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.231024    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/137f87ff-936c-417c-88f9-33c2a48d721f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.231967 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.231046    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-sysctl-conf\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.231967 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.231129    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-systemd\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.231967 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.231377    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-kubernetes\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.231967 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.231428    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/137f87ff-936c-417c-88f9-33c2a48d721f-cnibin\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.231967 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.231459    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-run\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.231967 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.231470    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-sysctl-d\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.231967 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.231492    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2c293f3c-09e5-4f86-991b-1613158f19ee-dbus\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:41.231967 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.231496    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-modprobe-d\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.231967 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.231537    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d39e8d10-0ae8-4265-9b99-ac2ac072b1ab-host\") pod \"node-ca-5frp2\" (UID: \"d39e8d10-0ae8-4265-9b99-ac2ac072b1ab\") " pod="openshift-image-registry/node-ca-5frp2"
Sep 11 13:48:41.231967 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.231772    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:41.231967 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.231936    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/137f87ff-936c-417c-88f9-33c2a48d721f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.232047    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-sysconfig\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.232073    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/137f87ff-936c-417c-88f9-33c2a48d721f-cni-binary-copy\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.232102    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-sys\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.232120    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/137f87ff-936c-417c-88f9-33c2a48d721f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.232142    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/137f87ff-936c-417c-88f9-33c2a48d721f-system-cni-dir\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.232152    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-host\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.232180    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2c293f3c-09e5-4f86-991b-1613158f19ee-kubelet-config\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.232204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-var-lib-kubelet\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.232299    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.232318    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d39e8d10-0ae8-4265-9b99-ac2ac072b1ab-serviceca\") pod \"node-ca-5frp2\" (UID: \"d39e8d10-0ae8-4265-9b99-ac2ac072b1ab\") " pod="openshift-image-registry/node-ca-5frp2"
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.232348    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-sysctl-conf\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.232372    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-lib-modules\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.232392    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb93b48e-5389-4693-af4a-52e1e2163832-konnectivity-ca\") pod \"konnectivity-agent-kdcfl\" (UID: \"cb93b48e-5389-4693-af4a-52e1e2163832\") " pod="kube-system/konnectivity-agent-kdcfl"
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.232428    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs podName:47db31ca-5cef-4c37-9da5-027f3df08815 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:41.73240216 +0000 UTC m=+3.129744892 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs") pod "network-metrics-daemon-59892" (UID: "47db31ca-5cef-4c37-9da5-027f3df08815") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.232463    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret podName:2c293f3c-09e5-4f86-991b-1613158f19ee nodeName:}" failed. No retries permitted until 2026-09-11 13:48:41.732452986 +0000 UTC m=+3.129795703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret") pod "global-pull-secret-syncer-hfjt6" (UID: "2c293f3c-09e5-4f86-991b-1613158f19ee") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:41.232532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.232468    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3896a84c-a6ea-468d-8e83-cded2f536e6d-lib-modules\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.234687 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.234580    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3896a84c-a6ea-468d-8e83-cded2f536e6d-tmp\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.234919 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.234898    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3896a84c-a6ea-468d-8e83-cded2f536e6d-etc-tuned\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.235334 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.235299    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb93b48e-5389-4693-af4a-52e1e2163832-agent-certs\") pod \"konnectivity-agent-kdcfl\" (UID: \"cb93b48e-5389-4693-af4a-52e1e2163832\") " pod="kube-system/konnectivity-agent-kdcfl"
Sep 11 13:48:41.238165 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.238141    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:48:41.238165 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.238167    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:48:41.238165 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.238183    2563 projected.go:196] Error preparing data for projected volume kube-api-access-7x97z for pod openshift-network-diagnostics/network-check-target-lrb9b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:41.238416 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.238261    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z podName:98c33f0c-094f-490f-af00-24b762506101 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:41.738239428 +0000 UTC m=+3.135582169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7x97z" (UniqueName: "kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z") pod "network-check-target-lrb9b" (UID: "98c33f0c-094f-490f-af00-24b762506101") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:41.241279 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.241208    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68kcp\" (UniqueName: \"kubernetes.io/projected/d39e8d10-0ae8-4265-9b99-ac2ac072b1ab-kube-api-access-68kcp\") pod \"node-ca-5frp2\" (UID: \"d39e8d10-0ae8-4265-9b99-ac2ac072b1ab\") " pod="openshift-image-registry/node-ca-5frp2"
Sep 11 13:48:41.241427 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.241346    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85429\" (UniqueName: \"kubernetes.io/projected/3896a84c-a6ea-468d-8e83-cded2f536e6d-kube-api-access-85429\") pod \"tuned-p7vp4\" (UID: \"3896a84c-a6ea-468d-8e83-cded2f536e6d\") " pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.241912 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.241888    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7t6h\" (UniqueName: \"kubernetes.io/projected/47db31ca-5cef-4c37-9da5-027f3df08815-kube-api-access-d7t6h\") pod \"network-metrics-daemon-59892\" (UID: \"47db31ca-5cef-4c37-9da5-027f3df08815\") " pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:41.243099 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.243072    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmv5r\" (UniqueName: \"kubernetes.io/projected/137f87ff-936c-417c-88f9-33c2a48d721f-kube-api-access-kmv5r\") pod \"multus-additional-cni-plugins-jkd4s\" (UID: \"137f87ff-936c-417c-88f9-33c2a48d721f\") " pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.317620 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.317573    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq"
Sep 11 13:48:41.325823 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.325788    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7q9dt"
Sep 11 13:48:41.335136 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.335105    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:48:41.344224 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.344195    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bc9vh"
Sep 11 13:48:41.353184 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.353151    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kwvws"
Sep 11 13:48:41.361556 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.361519    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kdcfl"
Sep 11 13:48:41.370770 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.370729    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jkd4s"
Sep 11 13:48:41.379996 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.379960    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5frp2"
Sep 11 13:48:41.387267 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.387223    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p7vp4"
Sep 11 13:48:41.737097 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.737056    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs\") pod \"network-metrics-daemon-59892\" (UID: \"47db31ca-5cef-4c37-9da5-027f3df08815\") " pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:41.737097 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.737101    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:41.737375 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.737225    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:41.737375 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.737225    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:41.737375 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.737293    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret podName:2c293f3c-09e5-4f86-991b-1613158f19ee nodeName:}" failed. No retries permitted until 2026-09-11 13:48:42.737272872 +0000 UTC m=+4.134615596 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret") pod "global-pull-secret-syncer-hfjt6" (UID: "2c293f3c-09e5-4f86-991b-1613158f19ee") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:41.737375 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.737312    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs podName:47db31ca-5cef-4c37-9da5-027f3df08815 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:42.737304907 +0000 UTC m=+4.134647624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs") pod "network-metrics-daemon-59892" (UID: "47db31ca-5cef-4c37-9da5-027f3df08815") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:41.837709 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:41.837667    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x97z\" (UniqueName: \"kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z\") pod \"network-check-target-lrb9b\" (UID: \"98c33f0c-094f-490f-af00-24b762506101\") " pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:41.837915 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.837835    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:48:41.837915 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.837868    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:48:41.837915 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.837881    2563 projected.go:196] Error preparing data for projected volume kube-api-access-7x97z for pod openshift-network-diagnostics/network-check-target-lrb9b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:41.838080 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:41.837958    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z podName:98c33f0c-094f-490f-af00-24b762506101 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:42.837936634 +0000 UTC m=+4.235279350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7x97z" (UniqueName: "kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z") pod "network-check-target-lrb9b" (UID: "98c33f0c-094f-490f-af00-24b762506101") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:42.056869 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:42.056764    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 13:43:41 +0000 UTC" deadline="2028-02-10 07:59:06.076736059 +0000 UTC"
Sep 11 13:48:42.056869 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:42.056808    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12402h10m24.019933541s"
Sep 11 13:48:42.125619 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:42.125575    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:42.125832 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:42.125760    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:48:42.173275 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:42.173169    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1675e538_4e61_4929_8935_2e31abb0299a.slice/crio-e69c3355cd4672fb89d2f301b4c5cd04b98b3858b9a8794d982878740951ecc5 WatchSource:0}: Error finding container e69c3355cd4672fb89d2f301b4c5cd04b98b3858b9a8794d982878740951ecc5: Status 404 returned error can't find the container with id e69c3355cd4672fb89d2f301b4c5cd04b98b3858b9a8794d982878740951ecc5
Sep 11 13:48:42.183026 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:42.182994    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1175fe53_1a2b_424b_a493_08c8d6ec865b.slice/crio-fb76cb3dafe9b08bac7b813a66a7c574b919e31f08b1a651c6912ad8bdb60573 WatchSource:0}: Error finding container fb76cb3dafe9b08bac7b813a66a7c574b919e31f08b1a651c6912ad8bdb60573: Status 404 returned error can't find the container with id fb76cb3dafe9b08bac7b813a66a7c574b919e31f08b1a651c6912ad8bdb60573
Sep 11 13:48:42.184911 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:42.184880    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod137f87ff_936c_417c_88f9_33c2a48d721f.slice/crio-8bea67855db717eeb9117b89abc5a7c9225bea6fa693f1c676641e8a2d2ddf69 WatchSource:0}: Error finding container 8bea67855db717eeb9117b89abc5a7c9225bea6fa693f1c676641e8a2d2ddf69: Status 404 returned error can't find the container with id 8bea67855db717eeb9117b89abc5a7c9225bea6fa693f1c676641e8a2d2ddf69
Sep 11 13:48:42.186125 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:42.186098    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4d48da9_3103_4bba_8652_483670f758b4.slice/crio-92c29e74a2ba9ad31d25ed8a07a51bfef748514d57aa13090d29033a6ada9b7d WatchSource:0}: Error finding container 92c29e74a2ba9ad31d25ed8a07a51bfef748514d57aa13090d29033a6ada9b7d: Status 404 returned error can't find the container with id 92c29e74a2ba9ad31d25ed8a07a51bfef748514d57aa13090d29033a6ada9b7d
Sep 11 13:48:42.186656 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:42.186620    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb93b48e_5389_4693_af4a_52e1e2163832.slice/crio-3dc154bb71b9c5b4d2c0bcb40087480994e09c1a631225f1606f2fb504eea510 WatchSource:0}: Error finding container 3dc154bb71b9c5b4d2c0bcb40087480994e09c1a631225f1606f2fb504eea510: Status 404 returned error can't find the container with id 3dc154bb71b9c5b4d2c0bcb40087480994e09c1a631225f1606f2fb504eea510
Sep 11 13:48:42.189334 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:42.188075    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3896a84c_a6ea_468d_8e83_cded2f536e6d.slice/crio-d1df5759f7f3ebe6e637128955c0bbc86fd030c8f2c72a8c245a034b2c6fc189 WatchSource:0}: Error finding container d1df5759f7f3ebe6e637128955c0bbc86fd030c8f2c72a8c245a034b2c6fc189: Status 404 returned error can't find the container with id d1df5759f7f3ebe6e637128955c0bbc86fd030c8f2c72a8c245a034b2c6fc189
Sep 11 13:48:42.189334 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:42.188596    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd39e8d10_0ae8_4265_9b99_ac2ac072b1ab.slice/crio-d39afdd50856484b8d7ce34bf63e6fbb86e5730cd0ec0afb14b8ca665e292ded WatchSource:0}: Error finding container d39afdd50856484b8d7ce34bf63e6fbb86e5730cd0ec0afb14b8ca665e292ded: Status 404 returned error can't find the container with id d39afdd50856484b8d7ce34bf63e6fbb86e5730cd0ec0afb14b8ca665e292ded
Sep 11 13:48:42.212735 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:48:42.212697    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a7cc659_3cac_4a00_9bb6_c90182f4c6c7.slice/crio-8fa9244873de754f90f306ca06b655ffa9a0b58e5220220edf493eccfe08289d WatchSource:0}: Error finding container 8fa9244873de754f90f306ca06b655ffa9a0b58e5220220edf493eccfe08289d: Status 404 returned error can't find the container with id 8fa9244873de754f90f306ca06b655ffa9a0b58e5220220edf493eccfe08289d
Sep 11 13:48:42.746329 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:42.746220    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs\") pod \"network-metrics-daemon-59892\" (UID: \"47db31ca-5cef-4c37-9da5-027f3df08815\") " pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:42.746329 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:42.746287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:42.746588 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:42.746442    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:42.746588 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:42.746550    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret podName:2c293f3c-09e5-4f86-991b-1613158f19ee nodeName:}" failed. No retries permitted until 2026-09-11 13:48:44.746526952 +0000 UTC m=+6.143869688 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret") pod "global-pull-secret-syncer-hfjt6" (UID: "2c293f3c-09e5-4f86-991b-1613158f19ee") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:42.747054 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:42.747029    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:42.747188 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:42.747095    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs podName:47db31ca-5cef-4c37-9da5-027f3df08815 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:44.747078325 +0000 UTC m=+6.144421058 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs") pod "network-metrics-daemon-59892" (UID: "47db31ca-5cef-4c37-9da5-027f3df08815") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:42.847760 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:42.847714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x97z\" (UniqueName: \"kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z\") pod \"network-check-target-lrb9b\" (UID: \"98c33f0c-094f-490f-af00-24b762506101\") " pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:42.847980 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:42.847945    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:48:42.847980 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:42.847967    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:48:42.847980 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:42.847979    2563 projected.go:196] Error preparing data for projected volume kube-api-access-7x97z for pod openshift-network-diagnostics/network-check-target-lrb9b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:42.848143 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:42.848045    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z podName:98c33f0c-094f-490f-af00-24b762506101 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:44.848025271 +0000 UTC m=+6.245368006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7x97z" (UniqueName: "kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z") pod "network-check-target-lrb9b" (UID: "98c33f0c-094f-490f-af00-24b762506101") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:43.058056 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.057905    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 13:43:41 +0000 UTC" deadline="2028-03-14 16:26:06.293600504 +0000 UTC"
Sep 11 13:48:43.058056 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.057978    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13202h37m23.235628029s"
Sep 11 13:48:43.128117 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.128078    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:43.128319 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:43.128243    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:48:43.128548 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.128416    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:43.128548 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:43.128506    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:48:43.152075 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.152015    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal" event={"ID":"fbf860b5b0097ab5f709024d70f9c3d5","Type":"ContainerStarted","Data":"15c406ce3d76ec37e90df6d4ccf7ad793ec93bf7e6ab2b1cf3c570f66bf8f095"}
Sep 11 13:48:43.175529 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.175482    2563 generic.go:356] "Generic (PLEG): container finished" podID="e80b9e5ab597f3b17c8733c311f97f1f" containerID="bf1bb2ad1c218a38d78cabe7531be2a4063b68b31bfa93c9cbb4e6a005ea2401" exitCode=0
Sep 11 13:48:43.175754 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.175671    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" event={"ID":"e80b9e5ab597f3b17c8733c311f97f1f","Type":"ContainerDied","Data":"bf1bb2ad1c218a38d78cabe7531be2a4063b68b31bfa93c9cbb4e6a005ea2401"}
Sep 11 13:48:43.188820 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.188780    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5frp2" event={"ID":"d39e8d10-0ae8-4265-9b99-ac2ac072b1ab","Type":"ContainerStarted","Data":"d39afdd50856484b8d7ce34bf63e6fbb86e5730cd0ec0afb14b8ca665e292ded"}
Sep 11 13:48:43.191344 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.191264    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal" podStartSLOduration=2.19123822 podStartE2EDuration="2.19123822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:43.16818702 +0000 UTC m=+4.565529765" watchObservedRunningTime="2026-09-11 13:48:43.19123822 +0000 UTC m=+4.588580963"
Sep 11 13:48:43.196182 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.195867    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kwvws" event={"ID":"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7","Type":"ContainerStarted","Data":"8fa9244873de754f90f306ca06b655ffa9a0b58e5220220edf493eccfe08289d"}
Sep 11 13:48:43.221372 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.220023    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p7vp4" event={"ID":"3896a84c-a6ea-468d-8e83-cded2f536e6d","Type":"ContainerStarted","Data":"d1df5759f7f3ebe6e637128955c0bbc86fd030c8f2c72a8c245a034b2c6fc189"}
Sep 11 13:48:43.248608 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.247853    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kdcfl" event={"ID":"cb93b48e-5389-4693-af4a-52e1e2163832","Type":"ContainerStarted","Data":"3dc154bb71b9c5b4d2c0bcb40087480994e09c1a631225f1606f2fb504eea510"}
Sep 11 13:48:43.251036 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.250984    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq" event={"ID":"e4d48da9-3103-4bba-8652-483670f758b4","Type":"ContainerStarted","Data":"92c29e74a2ba9ad31d25ed8a07a51bfef748514d57aa13090d29033a6ada9b7d"}
Sep 11 13:48:43.258387 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.258144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jkd4s" event={"ID":"137f87ff-936c-417c-88f9-33c2a48d721f","Type":"ContainerStarted","Data":"8bea67855db717eeb9117b89abc5a7c9225bea6fa693f1c676641e8a2d2ddf69"}
Sep 11 13:48:43.261178 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.261137    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bc9vh" event={"ID":"1175fe53-1a2b-424b-a493-08c8d6ec865b","Type":"ContainerStarted","Data":"fb76cb3dafe9b08bac7b813a66a7c574b919e31f08b1a651c6912ad8bdb60573"}
Sep 11 13:48:43.276358 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.276319    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7q9dt" event={"ID":"071b3477-ebf8-450b-9ca7-c2767dea40ea","Type":"ContainerStarted","Data":"cb92b1154b3edb819dbe46decb269f2c8c688c6e918ffb40b6b396ec8f359f93"}
Sep 11 13:48:43.298312 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:43.298261    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn" event={"ID":"1675e538-4e61-4929-8935-2e31abb0299a","Type":"ContainerStarted","Data":"e69c3355cd4672fb89d2f301b4c5cd04b98b3858b9a8794d982878740951ecc5"}
Sep 11 13:48:44.125357 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:44.125316    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:44.125957 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:44.125461    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:48:44.323938 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:44.323781    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" event={"ID":"e80b9e5ab597f3b17c8733c311f97f1f","Type":"ContainerStarted","Data":"626d5e391363421a9fe33b58efc7957c752ccaadc566d6781d585df190e72f1a"}
Sep 11 13:48:44.339982 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:44.339909    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" podStartSLOduration=3.339882828 podStartE2EDuration="3.339882828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:44.338974597 +0000 UTC m=+5.736317338" watchObservedRunningTime="2026-09-11 13:48:44.339882828 +0000 UTC m=+5.737225599"
Sep 11 13:48:44.763982 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:44.763936    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs\") pod \"network-metrics-daemon-59892\" (UID: \"47db31ca-5cef-4c37-9da5-027f3df08815\") " pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:44.764208 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:44.763998    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:44.764208 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:44.764181    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:44.764350 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:44.764249    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret podName:2c293f3c-09e5-4f86-991b-1613158f19ee nodeName:}" failed. No retries permitted until 2026-09-11 13:48:48.764229213 +0000 UTC m=+10.161572002 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret") pod "global-pull-secret-syncer-hfjt6" (UID: "2c293f3c-09e5-4f86-991b-1613158f19ee") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:44.764953 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:44.764749    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:44.764953 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:44.764813    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs podName:47db31ca-5cef-4c37-9da5-027f3df08815 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:48.764795521 +0000 UTC m=+10.162138245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs") pod "network-metrics-daemon-59892" (UID: "47db31ca-5cef-4c37-9da5-027f3df08815") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:44.865831 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:44.865059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x97z\" (UniqueName: \"kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z\") pod \"network-check-target-lrb9b\" (UID: \"98c33f0c-094f-490f-af00-24b762506101\") " pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:44.865831 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:44.865286    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:48:44.865831 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:44.865309    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:48:44.865831 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:44.865325    2563 projected.go:196] Error preparing data for projected volume kube-api-access-7x97z for pod openshift-network-diagnostics/network-check-target-lrb9b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:44.865831 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:44.865393    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z podName:98c33f0c-094f-490f-af00-24b762506101 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:48.865372149 +0000 UTC m=+10.262714914 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7x97z" (UniqueName: "kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z") pod "network-check-target-lrb9b" (UID: "98c33f0c-094f-490f-af00-24b762506101") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:45.126583 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:45.125963    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:45.126583 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:45.125899    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:45.126583 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:45.126133    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:48:45.126583 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:45.126235    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:48:46.125116 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:46.125071    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:46.125294 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:46.125225    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:48:47.125285 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:47.125244    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:47.125778 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:47.125302    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:47.125778 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:47.125430    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:48:47.125778 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:47.125601    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:48:48.125387 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:48.125293    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:48.125932 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:48.125431    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:48:48.800166 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:48.800117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs\") pod \"network-metrics-daemon-59892\" (UID: \"47db31ca-5cef-4c37-9da5-027f3df08815\") " pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:48.800392 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:48.800185    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:48.800392 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:48.800332    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:48.800560 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:48.800403    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret podName:2c293f3c-09e5-4f86-991b-1613158f19ee nodeName:}" failed. No retries permitted until 2026-09-11 13:48:56.800382178 +0000 UTC m=+18.197724909 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret") pod "global-pull-secret-syncer-hfjt6" (UID: "2c293f3c-09e5-4f86-991b-1613158f19ee") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:48.800881 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:48.800854    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:48.800968 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:48.800915    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs podName:47db31ca-5cef-4c37-9da5-027f3df08815 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:56.800898317 +0000 UTC m=+18.198241081 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs") pod "network-metrics-daemon-59892" (UID: "47db31ca-5cef-4c37-9da5-027f3df08815") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:48.901071 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:48.901027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x97z\" (UniqueName: \"kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z\") pod \"network-check-target-lrb9b\" (UID: \"98c33f0c-094f-490f-af00-24b762506101\") " pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:48.901302 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:48.901224    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:48:48.901302 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:48.901248    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:48:48.901302 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:48.901262    2563 projected.go:196] Error preparing data for projected volume kube-api-access-7x97z for pod openshift-network-diagnostics/network-check-target-lrb9b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:48.901476 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:48.901327    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z podName:98c33f0c-094f-490f-af00-24b762506101 nodeName:}" failed. No retries permitted until 2026-09-11 13:48:56.90131268 +0000 UTC m=+18.298655397 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7x97z" (UniqueName: "kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z") pod "network-check-target-lrb9b" (UID: "98c33f0c-094f-490f-af00-24b762506101") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:49.127224 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:49.127120    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:49.128037 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:49.127844    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:49.128322 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:49.128291    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:48:49.128806 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:49.128421    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:48:50.125661 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:50.125602    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:50.125865 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:50.125761    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:48:51.125256 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:51.125218    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:51.125863 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:51.125379    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:48:51.125956 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:51.125862    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:51.126052 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:51.125993    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:48:52.125480 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:52.125441    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:52.125914 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:52.125547    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:48:53.125018 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:53.124981    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:53.125203 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:53.125123    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:48:53.125517 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:53.124981    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:53.125913 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:53.125646    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:48:54.125357 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:54.125257    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:54.125531 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:54.125393    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:48:55.125483 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:55.125441    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:55.125686 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:55.125580    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:48:55.125686 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:55.125641    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:55.126035 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:55.125764    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:48:56.124878 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:56.124839    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:56.125140 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:56.124975    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:48:56.864744 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:56.864697    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs\") pod \"network-metrics-daemon-59892\" (UID: \"47db31ca-5cef-4c37-9da5-027f3df08815\") " pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:56.865193 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:56.864756    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:56.865193 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:56.864861    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:56.865193 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:56.864900    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:56.865193 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:56.864954    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs podName:47db31ca-5cef-4c37-9da5-027f3df08815 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:12.86493353 +0000 UTC m=+34.262276295 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs") pod "network-metrics-daemon-59892" (UID: "47db31ca-5cef-4c37-9da5-027f3df08815") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:48:56.865193 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:56.864974    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret podName:2c293f3c-09e5-4f86-991b-1613158f19ee nodeName:}" failed. No retries permitted until 2026-09-11 13:49:12.864966745 +0000 UTC m=+34.262309461 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret") pod "global-pull-secret-syncer-hfjt6" (UID: "2c293f3c-09e5-4f86-991b-1613158f19ee") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:48:56.965477 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:56.965432    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x97z\" (UniqueName: \"kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z\") pod \"network-check-target-lrb9b\" (UID: \"98c33f0c-094f-490f-af00-24b762506101\") " pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:56.965674 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:56.965647    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:48:56.965726 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:56.965675    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:48:56.965726 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:56.965690    2563 projected.go:196] Error preparing data for projected volume kube-api-access-7x97z for pod openshift-network-diagnostics/network-check-target-lrb9b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:56.965809 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:56.965759    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z podName:98c33f0c-094f-490f-af00-24b762506101 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:12.9657402 +0000 UTC m=+34.363082924 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7x97z" (UniqueName: "kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z") pod "network-check-target-lrb9b" (UID: "98c33f0c-094f-490f-af00-24b762506101") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:48:57.125081 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:57.124988    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:57.125294 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:57.124988    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:57.125294 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:57.125141    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:48:57.125294 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:57.125213    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:48:58.125038 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:58.124999    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:48:58.125506 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:58.125109    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:48:59.128049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:59.128020    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:48:59.128508 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:48:59.128061    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:48:59.128508 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:59.128143    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:48:59.128508 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:48:59.128264    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:49:00.125367 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:00.125326    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:49:00.125570 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:00.125450    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:49:01.125099 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:01.125056    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:49:01.125670 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:01.125224    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:49:01.125670 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:01.125264    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:49:01.125670 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:01.125360    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:49:02.125542 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.125271    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:49:02.125542 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:02.125522    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:49:02.392122 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.391857    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5frp2" event={"ID":"d39e8d10-0ae8-4265-9b99-ac2ac072b1ab","Type":"ContainerStarted","Data":"5749d023918720e273423f373663648683355b5cc923b39c2b24a47667421379"}
Sep 11 13:49:02.393667 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.393617    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kwvws" event={"ID":"4a7cc659-3cac-4a00-9bb6-c90182f4c6c7","Type":"ContainerStarted","Data":"a68eeedfb0d1f4eb49d9e616dd610bc9ff8f25cf3f984deff99c635380514388"}
Sep 11 13:49:02.395420 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.395384    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p7vp4" event={"ID":"3896a84c-a6ea-468d-8e83-cded2f536e6d","Type":"ContainerStarted","Data":"8c3dac88cc5b23b8adb37b93b245d243bd00785f6cfe15ee83a3c401b3b47e00"}
Sep 11 13:49:02.397218 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.397192    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kdcfl" event={"ID":"cb93b48e-5389-4693-af4a-52e1e2163832","Type":"ContainerStarted","Data":"0de1978a61ba01ad4541b4d3a1081015d0ad83409e60a68716f1e79c5c17c72c"}
Sep 11 13:49:02.398812 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.398788    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq" event={"ID":"e4d48da9-3103-4bba-8652-483670f758b4","Type":"ContainerStarted","Data":"994a52f93d698ef48792069872ced3805bc8f7da05a93d86d8fee1c66dfb572c"}
Sep 11 13:49:02.400548 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.400527    2563 generic.go:356] "Generic (PLEG): container finished" podID="137f87ff-936c-417c-88f9-33c2a48d721f" containerID="ff60f1df960dfe65bf24f8d4f114b7868e59716f6cee82b3ccaa9d30338ff005" exitCode=0
Sep 11 13:49:02.400676 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.400594    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jkd4s" event={"ID":"137f87ff-936c-417c-88f9-33c2a48d721f","Type":"ContainerDied","Data":"ff60f1df960dfe65bf24f8d4f114b7868e59716f6cee82b3ccaa9d30338ff005"}
Sep 11 13:49:02.402294 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.402270    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7q9dt" event={"ID":"071b3477-ebf8-450b-9ca7-c2767dea40ea","Type":"ContainerStarted","Data":"6e8b41febed873f1ab5fa7a1e59685e5bdaafbfbd1d5cfe514cc28c8bedad1ad"}
Sep 11 13:49:02.405673 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.405603    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5frp2" podStartSLOduration=3.375797467 podStartE2EDuration="22.405589727s" totalImagesPullingTime="19.02979226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:02.404728231 +0000 UTC m=+23.802070967" watchObservedRunningTime="2026-09-11 13:49:02.405589727 +0000 UTC m=+23.802932465"
Sep 11 13:49:02.406370 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.406345    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn" event={"ID":"1675e538-4e61-4929-8935-2e31abb0299a","Type":"ContainerStarted","Data":"5bf166f8b4e4a0ae25fdcb43c33f4dd1c793b84922244e347f4197f84605137c"}
Sep 11 13:49:02.406474 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.406426    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn" event={"ID":"1675e538-4e61-4929-8935-2e31abb0299a","Type":"ContainerStarted","Data":"911f9e50ec6b9cff4295922d21c0e99f8900432a9449914cf8e94ce336237459"}
Sep 11 13:49:02.406525 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.406475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn" event={"ID":"1675e538-4e61-4929-8935-2e31abb0299a","Type":"ContainerStarted","Data":"633ae4d0cb2cc233d31b16d51e905c17756f29872ac4ddb8d20ceee3b4ec6671"}
Sep 11 13:49:02.406564 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.406524    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn" event={"ID":"1675e538-4e61-4929-8935-2e31abb0299a","Type":"ContainerStarted","Data":"b17d67cc0de97a040a885f92fb204838b62d5990a17d307a2be2ce8cef2ae3ad"}
Sep 11 13:49:02.406608 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.406571    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn" event={"ID":"1675e538-4e61-4929-8935-2e31abb0299a","Type":"ContainerStarted","Data":"ee04fffc2bcae12ce25becc54d9c2696afe0284ebebd8bb94ca9c7c8074b1910"}
Sep 11 13:49:02.406681 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.406616    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn" event={"ID":"1675e538-4e61-4929-8935-2e31abb0299a","Type":"ContainerStarted","Data":"48eb819a538f5d1ac5d0f4fcf0c2bcf2598c602648904eccfb5e529ee270986e"}
Sep 11 13:49:02.418556 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.418503    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kdcfl" podStartSLOduration=4.394415687 podStartE2EDuration="23.418489543s" totalImagesPullingTime="19.024073856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:02.418243985 +0000 UTC m=+23.815586737" watchObservedRunningTime="2026-09-11 13:49:02.418489543 +0000 UTC m=+23.815832279"
Sep 11 13:49:02.449558 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.449502    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7q9dt" podStartSLOduration=4.382335939 podStartE2EDuration="23.449485332s" totalImagesPullingTime="19.067149393s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:02.449136754 +0000 UTC m=+23.846479490" watchObservedRunningTime="2026-09-11 13:49:02.449485332 +0000 UTC m=+23.846828068"
Sep 11 13:49:02.465814 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.465761    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kwvws" podStartSLOduration=4.385578052 podStartE2EDuration="23.465744131s" totalImagesPullingTime="19.080166079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:02.465173396 +0000 UTC m=+23.862516153" watchObservedRunningTime="2026-09-11 13:49:02.465744131 +0000 UTC m=+23.863086868"
Sep 11 13:49:02.481935 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.481872    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-p7vp4" podStartSLOduration=3.453423464 podStartE2EDuration="22.48185379s" totalImagesPullingTime="19.028430326s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:02.481247721 +0000 UTC m=+23.878590491" watchObservedRunningTime="2026-09-11 13:49:02.48185379 +0000 UTC m=+23.879196525"
Sep 11 13:49:02.604668 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.604556    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kdcfl_cb93b48e-5389-4693-af4a-52e1e2163832/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:02.686012 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.685980    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kdcfl_cb93b48e-5389-4693-af4a-52e1e2163832/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:02.853148 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:02.853126    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 13:49:03.056072 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:03.055990    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kdcfl_cb93b48e-5389-4693-af4a-52e1e2163832/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:03.099301 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:03.099193    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T13:49:02.853143045Z","UUID":"b32d918f-9d9f-4c5f-850b-22508c5c37f3","Handler":null,"Name":"","Endpoint":""}
Sep 11 13:49:03.100746 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:03.100727    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 13:49:03.100883 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:03.100756    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 13:49:03.128674 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:03.128650    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:49:03.129095 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:03.128738    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:49:03.129095 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:03.128795    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:49:03.129095 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:03.128889    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:49:03.156636 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:03.156592    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kdcfl_cb93b48e-5389-4693-af4a-52e1e2163832/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:03.412152 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:03.412067    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq" event={"ID":"e4d48da9-3103-4bba-8652-483670f758b4","Type":"ContainerStarted","Data":"e5281b3772fa09c6f34b788482dc271bde03ddebb6c9e4661724e449a80b593e"}
Sep 11 13:49:03.414104 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:03.414070    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bc9vh" event={"ID":"1175fe53-1a2b-424b-a493-08c8d6ec865b","Type":"ContainerStarted","Data":"3a841a10bf8f3a8d587244c081af942a14a3c98c7f3cddf5e0edcc11648b0d42"}
Sep 11 13:49:03.435889 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:03.435823    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bc9vh" podStartSLOduration=5.379642072 podStartE2EDuration="24.435802289s" totalImagesPullingTime="19.056160217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:03.43561795 +0000 UTC m=+24.832960689" watchObservedRunningTime="2026-09-11 13:49:03.435802289 +0000 UTC m=+24.833145027"
Sep 11 13:49:03.584738 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:03.584621    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kdcfl_cb93b48e-5389-4693-af4a-52e1e2163832/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:03.663935 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:03.663821    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kdcfl_cb93b48e-5389-4693-af4a-52e1e2163832/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:03.728412 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:03.728370    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kdcfl_cb93b48e-5389-4693-af4a-52e1e2163832/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:03.936100 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:03.936066    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kdcfl_cb93b48e-5389-4693-af4a-52e1e2163832/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:04.037081 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:04.037040    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kdcfl_cb93b48e-5389-4693-af4a-52e1e2163832/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:04.125616 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:04.125578    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:49:04.125762 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:04.125733    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:49:04.420657 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:04.420605    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn" event={"ID":"1675e538-4e61-4929-8935-2e31abb0299a","Type":"ContainerStarted","Data":"2bb2ad3ba193a48a04e90480dbc9399d998c2144da5a39c0fd7bb662002b9dc1"}
Sep 11 13:49:04.423166 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:04.423124    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq" event={"ID":"e4d48da9-3103-4bba-8652-483670f758b4","Type":"ContainerStarted","Data":"439b84e2aabcd2826532e4bfd8463ee952381a73fdd918dba62832afac7674f4"}
Sep 11 13:49:04.442502 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:04.442402    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zcvlq" podStartSLOduration=4.122939329 podStartE2EDuration="25.44238068s" totalImagesPullingTime="21.319441351s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:04.441840294 +0000 UTC m=+25.839183035" watchObservedRunningTime="2026-09-11 13:49:04.44238068 +0000 UTC m=+25.839723417"
Sep 11 13:49:05.125159 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:05.125122    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:49:05.125354 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:05.125254    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:49:05.125354 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:05.125309    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:49:05.125462 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:05.125430    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:49:05.395161 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:05.395070    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kdcfl"
Sep 11 13:49:05.395708 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:05.395682    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kdcfl"
Sep 11 13:49:06.124957 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:06.124918    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:49:06.125383 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:06.125070    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:49:07.125239 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:07.125202    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:49:07.125691 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:07.125202    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:49:07.125691 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:07.125348    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:49:07.126221 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:07.126170    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:49:07.435928 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:07.435691    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn" event={"ID":"1675e538-4e61-4929-8935-2e31abb0299a","Type":"ContainerStarted","Data":"66f27de116390e1159164f69c38eb23426bf7efc51aec98d99709ec6e121bc60"}
Sep 11 13:49:07.436187 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:07.436164    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:49:07.436855 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:07.436242    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:49:07.436855 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:07.436292    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:49:07.438405 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:07.438359    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jkd4s" event={"ID":"137f87ff-936c-417c-88f9-33c2a48d721f","Type":"ContainerStarted","Data":"b7471342484d1922eb477294e58dc9d9fa86725c722bdb1952c0f0e422d92825"}
Sep 11 13:49:07.453692 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:07.453663    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:49:07.454088 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:07.454068    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:49:07.463399 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:07.463348    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn" podStartSLOduration=9.351149427 podStartE2EDuration="28.463333933s" totalImagesPullingTime="19.112184506s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:07.462375186 +0000 UTC m=+28.859717923" watchObservedRunningTime="2026-09-11 13:49:07.463333933 +0000 UTC m=+28.860676670"
Sep 11 13:49:08.125518 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:08.125476    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:49:08.126283 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:08.125606    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:49:08.442761 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:08.442672    2563 generic.go:356] "Generic (PLEG): container finished" podID="137f87ff-936c-417c-88f9-33c2a48d721f" containerID="b7471342484d1922eb477294e58dc9d9fa86725c722bdb1952c0f0e422d92825" exitCode=0
Sep 11 13:49:08.442913 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:08.442763    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jkd4s" event={"ID":"137f87ff-936c-417c-88f9-33c2a48d721f","Type":"ContainerDied","Data":"b7471342484d1922eb477294e58dc9d9fa86725c722bdb1952c0f0e422d92825"}
Sep 11 13:49:09.125945 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.125767    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:49:09.126394 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.125827    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:49:09.126394 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:09.126042    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:49:09.126394 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:09.126109    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:49:09.230743 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.230711    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lrb9b"]
Sep 11 13:49:09.242443 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.242366    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hfjt6"]
Sep 11 13:49:09.242580 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.242485    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:49:09.242615 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:09.242584    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:49:09.250238 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.250192    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-59892"]
Sep 11 13:49:09.333218 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.333188    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kdcfl_cb93b48e-5389-4693-af4a-52e1e2163832/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:09.406435 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.406402    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kdcfl_cb93b48e-5389-4693-af4a-52e1e2163832/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:09.444972 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.444940    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:49:09.445116 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.444940    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:49:09.445116 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:09.445058    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:49:09.445184 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:09.445129    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:49:09.468538 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.468506    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kdcfl_cb93b48e-5389-4693-af4a-52e1e2163832/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:09.523727 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.523644    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kdcfl_cb93b48e-5389-4693-af4a-52e1e2163832/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:09.640157 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.640133    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kdcfl"
Sep 11 13:49:09.640341 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.640316    2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 13:49:09.640938 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.640912    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kdcfl"
Sep 11 13:49:09.712443 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:09.712409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kdcfl_cb93b48e-5389-4693-af4a-52e1e2163832/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:10.449739 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:10.449705    2563 generic.go:356] "Generic (PLEG): container finished" podID="137f87ff-936c-417c-88f9-33c2a48d721f" containerID="947d908276809c2facaa3bd0742f9be0eccf712a92aacabdd29283bb1137cb22" exitCode=0
Sep 11 13:49:10.450169 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:10.449780    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jkd4s" event={"ID":"137f87ff-936c-417c-88f9-33c2a48d721f","Type":"ContainerDied","Data":"947d908276809c2facaa3bd0742f9be0eccf712a92aacabdd29283bb1137cb22"}
Sep 11 13:49:11.125091 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:11.125049    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:49:11.125240 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:11.125100    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:49:11.125240 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:11.125050    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:49:11.125240 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:11.125197    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:49:11.125366 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:11.125252    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:49:11.125366 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:11.125325    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:49:11.454167 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:11.454132    2563 generic.go:356] "Generic (PLEG): container finished" podID="137f87ff-936c-417c-88f9-33c2a48d721f" containerID="f58353e8bf51c9c8624c0bf9aa92442a006894b3c454df3bd5e66ee773e1f2b3" exitCode=0
Sep 11 13:49:11.454500 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:11.454190    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jkd4s" event={"ID":"137f87ff-936c-417c-88f9-33c2a48d721f","Type":"ContainerDied","Data":"f58353e8bf51c9c8624c0bf9aa92442a006894b3c454df3bd5e66ee773e1f2b3"}
Sep 11 13:49:12.885615 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:12.885386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs\") pod \"network-metrics-daemon-59892\" (UID: \"47db31ca-5cef-4c37-9da5-027f3df08815\") " pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:49:12.886068 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:12.885523    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:49:12.886068 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:12.885656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:49:12.886068 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:12.885727    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs podName:47db31ca-5cef-4c37-9da5-027f3df08815 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:44.885702617 +0000 UTC m=+66.283045337 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs") pod "network-metrics-daemon-59892" (UID: "47db31ca-5cef-4c37-9da5-027f3df08815") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:49:12.886068 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:12.885783    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:49:12.886068 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:12.885834    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret podName:2c293f3c-09e5-4f86-991b-1613158f19ee nodeName:}" failed. No retries permitted until 2026-09-11 13:49:44.885819301 +0000 UTC m=+66.283162016 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret") pod "global-pull-secret-syncer-hfjt6" (UID: "2c293f3c-09e5-4f86-991b-1613158f19ee") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:49:12.987047 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:12.987009    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x97z\" (UniqueName: \"kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z\") pod \"network-check-target-lrb9b\" (UID: \"98c33f0c-094f-490f-af00-24b762506101\") " pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:49:12.987228 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:12.987159    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:49:12.987228 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:12.987179    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:49:12.987228 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:12.987194    2563 projected.go:196] Error preparing data for projected volume kube-api-access-7x97z for pod openshift-network-diagnostics/network-check-target-lrb9b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:49:12.987387 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:12.987260    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z podName:98c33f0c-094f-490f-af00-24b762506101 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:44.987236243 +0000 UTC m=+66.384578967 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7x97z" (UniqueName: "kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z") pod "network-check-target-lrb9b" (UID: "98c33f0c-094f-490f-af00-24b762506101") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:49:13.125229 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.125181    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:49:13.125229 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.125181    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:49:13.125449 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.125327    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:49:13.125449 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:13.125330    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrb9b" podUID="98c33f0c-094f-490f-af00-24b762506101"
Sep 11 13:49:13.125526 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:13.125489    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-59892" podUID="47db31ca-5cef-4c37-9da5-027f3df08815"
Sep 11 13:49:13.125603 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:13.125561    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hfjt6" podUID="2c293f3c-09e5-4f86-991b-1613158f19ee"
Sep 11 13:49:13.877916 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.877886    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeReady"
Sep 11 13:49:13.878072 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.878028    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 13:49:13.911125 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.911080    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cfbdd748c-l979v"]
Sep 11 13:49:13.923184 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.923156    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rxc9q"]
Sep 11 13:49:13.923463 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.923436    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:13.927359 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.927326    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 13:49:13.927359 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.927363    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 13:49:13.927652 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.927488    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8k7zz\""
Sep 11 13:49:13.928541 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.928521    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 13:49:13.933295 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.933273    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 13:49:13.939493 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.939471    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cfbdd748c-l979v"]
Sep 11 13:49:13.939605 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.939510    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8cwcd"]
Sep 11 13:49:13.939843 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.939638    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:13.941798 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.941777    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-q92jh\""
Sep 11 13:49:13.942291 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.942273    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 13:49:13.942705 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.942488    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 13:49:13.950558 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.950533    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rxc9q"]
Sep 11 13:49:13.950696 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.950577    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8cwcd"]
Sep 11 13:49:13.950696 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.950684    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8cwcd"
Sep 11 13:49:13.952573 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.952551    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 13:49:13.952685 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.952589    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 13:49:13.952685 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.952606    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xnqjg\""
Sep 11 13:49:13.952685 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.952563    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 13:49:13.993038 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.992995    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sxxp\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-kube-api-access-4sxxp\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:13.993038 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.993036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:13.993287 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.993066    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-certificates\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:13.993287 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.993156    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fac2fe28-c77e-4648-b789-b18824e80a3f-ca-trust-extracted\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:13.993287 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.993199    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fac2fe28-c77e-4648-b789-b18824e80a3f-image-registry-private-configuration\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:13.993287 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.993232    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-bound-sa-token\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:13.993287 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.993249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fac2fe28-c77e-4648-b789-b18824e80a3f-trusted-ca\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:13.993287 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:13.993270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fac2fe28-c77e-4648-b789-b18824e80a3f-installation-pull-secrets\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.093738 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.093648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fac2fe28-c77e-4648-b789-b18824e80a3f-image-registry-private-configuration\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.093738 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.093698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-bound-sa-token\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.093738 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.093720    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fac2fe28-c77e-4648-b789-b18824e80a3f-trusted-ca\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.093738 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.093739    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fac2fe28-c77e-4648-b789-b18824e80a3f-installation-pull-secrets\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.093991 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.093760    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vw5j\" (UniqueName: \"kubernetes.io/projected/2054806b-ed98-4a9c-b889-0a4efda17d75-kube-api-access-2vw5j\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:14.093991 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.093784    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2054806b-ed98-4a9c-b889-0a4efda17d75-config-volume\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:14.093991 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.093814    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sxxp\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-kube-api-access-4sxxp\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.093991 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.093840    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.093991 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.093868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-certificates\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.093991 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.093891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:14.093991 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.093914    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2054806b-ed98-4a9c-b889-0a4efda17d75-tmp-dir\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:14.093991 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.093929    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3d7f55a-5d08-4e04-a09c-73c237ab55e8-cert\") pod \"ingress-canary-8cwcd\" (UID: \"a3d7f55a-5d08-4e04-a09c-73c237ab55e8\") " pod="openshift-ingress-canary/ingress-canary-8cwcd"
Sep 11 13:49:14.093991 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.093944    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhdvc\" (UniqueName: \"kubernetes.io/projected/a3d7f55a-5d08-4e04-a09c-73c237ab55e8-kube-api-access-nhdvc\") pod \"ingress-canary-8cwcd\" (UID: \"a3d7f55a-5d08-4e04-a09c-73c237ab55e8\") " pod="openshift-ingress-canary/ingress-canary-8cwcd"
Sep 11 13:49:14.094304 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.093999    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fac2fe28-c77e-4648-b789-b18824e80a3f-ca-trust-extracted\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.095723 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.094398    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fac2fe28-c77e-4648-b789-b18824e80a3f-ca-trust-extracted\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.095723 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:14.094715    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:49:14.095723 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:14.094734    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfbdd748c-l979v: secret "image-registry-tls" not found
Sep 11 13:49:14.095723 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:14.094795    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls podName:fac2fe28-c77e-4648-b789-b18824e80a3f nodeName:}" failed. No retries permitted until 2026-09-11 13:49:14.594774455 +0000 UTC m=+35.992117170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls") pod "image-registry-6cfbdd748c-l979v" (UID: "fac2fe28-c77e-4648-b789-b18824e80a3f") : secret "image-registry-tls" not found
Sep 11 13:49:14.095723 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.095309    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-certificates\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.095723 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.095681    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fac2fe28-c77e-4648-b789-b18824e80a3f-trusted-ca\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.099106 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.099080    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fac2fe28-c77e-4648-b789-b18824e80a3f-image-registry-private-configuration\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.099231 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.099131    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fac2fe28-c77e-4648-b789-b18824e80a3f-installation-pull-secrets\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.103426 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.103374    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-bound-sa-token\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.104305 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.104270    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sxxp\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-kube-api-access-4sxxp\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.194676 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.194613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vw5j\" (UniqueName: \"kubernetes.io/projected/2054806b-ed98-4a9c-b889-0a4efda17d75-kube-api-access-2vw5j\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:14.194676 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.194682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2054806b-ed98-4a9c-b889-0a4efda17d75-config-volume\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:14.194930 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.194738    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:14.194930 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.194765    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2054806b-ed98-4a9c-b889-0a4efda17d75-tmp-dir\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:14.194930 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.194882    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3d7f55a-5d08-4e04-a09c-73c237ab55e8-cert\") pod \"ingress-canary-8cwcd\" (UID: \"a3d7f55a-5d08-4e04-a09c-73c237ab55e8\") " pod="openshift-ingress-canary/ingress-canary-8cwcd"
Sep 11 13:49:14.194930 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:14.194919    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:49:14.194930 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.194930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhdvc\" (UniqueName: \"kubernetes.io/projected/a3d7f55a-5d08-4e04-a09c-73c237ab55e8-kube-api-access-nhdvc\") pod \"ingress-canary-8cwcd\" (UID: \"a3d7f55a-5d08-4e04-a09c-73c237ab55e8\") " pod="openshift-ingress-canary/ingress-canary-8cwcd"
Sep 11 13:49:14.195169 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:14.194991    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls podName:2054806b-ed98-4a9c-b889-0a4efda17d75 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:14.69496729 +0000 UTC m=+36.092310010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls") pod "dns-default-rxc9q" (UID: "2054806b-ed98-4a9c-b889-0a4efda17d75") : secret "dns-default-metrics-tls" not found
Sep 11 13:49:14.195169 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.195130    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2054806b-ed98-4a9c-b889-0a4efda17d75-tmp-dir\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:14.195434 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.195379    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2054806b-ed98-4a9c-b889-0a4efda17d75-config-volume\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:14.197804 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.197782    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3d7f55a-5d08-4e04-a09c-73c237ab55e8-cert\") pod \"ingress-canary-8cwcd\" (UID: \"a3d7f55a-5d08-4e04-a09c-73c237ab55e8\") " pod="openshift-ingress-canary/ingress-canary-8cwcd"
Sep 11 13:49:14.203022 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.202998    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vw5j\" (UniqueName: \"kubernetes.io/projected/2054806b-ed98-4a9c-b889-0a4efda17d75-kube-api-access-2vw5j\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:14.203580 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.203551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhdvc\" (UniqueName: \"kubernetes.io/projected/a3d7f55a-5d08-4e04-a09c-73c237ab55e8-kube-api-access-nhdvc\") pod \"ingress-canary-8cwcd\" (UID: \"a3d7f55a-5d08-4e04-a09c-73c237ab55e8\") " pod="openshift-ingress-canary/ingress-canary-8cwcd"
Sep 11 13:49:14.259064 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.259025    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8cwcd"
Sep 11 13:49:14.599105 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.599064    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:14.599355 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:14.599224    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:49:14.599355 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:14.599243    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfbdd748c-l979v: secret "image-registry-tls" not found
Sep 11 13:49:14.599355 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:14.599313    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls podName:fac2fe28-c77e-4648-b789-b18824e80a3f nodeName:}" failed. No retries permitted until 2026-09-11 13:49:15.599291339 +0000 UTC m=+36.996634060 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls") pod "image-registry-6cfbdd748c-l979v" (UID: "fac2fe28-c77e-4648-b789-b18824e80a3f") : secret "image-registry-tls" not found
Sep 11 13:49:14.700649 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:14.700543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:14.700820 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:14.700696    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:49:14.700820 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:14.700764    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls podName:2054806b-ed98-4a9c-b889-0a4efda17d75 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:15.700746997 +0000 UTC m=+37.098089728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls") pod "dns-default-rxc9q" (UID: "2054806b-ed98-4a9c-b889-0a4efda17d75") : secret "dns-default-metrics-tls" not found
Sep 11 13:49:15.125590 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:15.125550    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:49:15.126171 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:15.125550    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:49:15.126171 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:15.125550    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:49:15.128525 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:15.128500    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:49:15.128833 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:15.128818    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6pjch\""
Sep 11 13:49:15.129318 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:15.128920    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:49:15.129318 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:15.129053    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:49:15.129318 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:15.129132    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:49:15.129318 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:15.129261    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mlc2t\""
Sep 11 13:49:15.608484 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:15.608444    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:15.608673 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:15.608601    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:49:15.608673 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:15.608644    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfbdd748c-l979v: secret "image-registry-tls" not found
Sep 11 13:49:15.608796 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:15.608702    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls podName:fac2fe28-c77e-4648-b789-b18824e80a3f nodeName:}" failed. No retries permitted until 2026-09-11 13:49:17.608686957 +0000 UTC m=+39.006029671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls") pod "image-registry-6cfbdd748c-l979v" (UID: "fac2fe28-c77e-4648-b789-b18824e80a3f") : secret "image-registry-tls" not found
Sep 11 13:49:15.709846 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:15.709802    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:15.710010 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:15.709980    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:49:15.710059 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:15.710051    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls podName:2054806b-ed98-4a9c-b889-0a4efda17d75 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:17.710034701 +0000 UTC m=+39.107377419 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls") pod "dns-default-rxc9q" (UID: "2054806b-ed98-4a9c-b889-0a4efda17d75") : secret "dns-default-metrics-tls" not found
Sep 11 13:49:15.753999 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:15.753966    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8cwcd"]
Sep 11 13:49:15.769728 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:49:15.769698    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3d7f55a_5d08_4e04_a09c_73c237ab55e8.slice/crio-6c18e615ee60dd5b45e368a00f20c6a8a14cd76e9c4a4c986434df42b697bcb8 WatchSource:0}: Error finding container 6c18e615ee60dd5b45e368a00f20c6a8a14cd76e9c4a4c986434df42b697bcb8: Status 404 returned error can't find the container with id 6c18e615ee60dd5b45e368a00f20c6a8a14cd76e9c4a4c986434df42b697bcb8
Sep 11 13:49:16.467356 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:16.467317    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8cwcd" event={"ID":"a3d7f55a-5d08-4e04-a09c-73c237ab55e8","Type":"ContainerStarted","Data":"6c18e615ee60dd5b45e368a00f20c6a8a14cd76e9c4a4c986434df42b697bcb8"}
Sep 11 13:49:16.469990 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:16.469958    2563 generic.go:356] "Generic (PLEG): container finished" podID="137f87ff-936c-417c-88f9-33c2a48d721f" containerID="3cecc91057ab8359fb8f5e9dbf16f6a20f69adcb2fc43f550908c5c9c468f43a" exitCode=0
Sep 11 13:49:16.470116 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:16.470036    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jkd4s" event={"ID":"137f87ff-936c-417c-88f9-33c2a48d721f","Type":"ContainerDied","Data":"3cecc91057ab8359fb8f5e9dbf16f6a20f69adcb2fc43f550908c5c9c468f43a"}
Sep 11 13:49:16.898216 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:16.897983    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pzd6j"]
Sep 11 13:49:16.911755 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:16.911720    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:16.915534 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:16.915495    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 13:49:16.916323 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:16.916116    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 13:49:16.916323 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:16.916269    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 13:49:16.916502 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:16.916434    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 13:49:16.916988 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:16.916965    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-klbks\""
Sep 11 13:49:16.925296 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:16.925271    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pzd6j"]
Sep 11 13:49:17.021503 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.021465    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/82fbbb40-345c-41b3-ac15-49360dcd7859-data-volume\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:17.021503 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.021507    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:17.021772 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.021536    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgqvj\" (UniqueName: \"kubernetes.io/projected/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-api-access-jgqvj\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:17.021772 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.021619    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/82fbbb40-345c-41b3-ac15-49360dcd7859-crio-socket\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:17.021772 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.021699    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:17.123112 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.123079    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/82fbbb40-345c-41b3-ac15-49360dcd7859-crio-socket\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:17.123468 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.123123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:17.123900 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.123601    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/82fbbb40-345c-41b3-ac15-49360dcd7859-data-volume\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:17.123900 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.123674    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:17.123900 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.123687    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/82fbbb40-345c-41b3-ac15-49360dcd7859-crio-socket\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:17.123900 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:17.123741    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:17.123900 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:17.123778    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:49:17.123900 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:17.123830    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls podName:82fbbb40-345c-41b3-ac15-49360dcd7859 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:17.623807536 +0000 UTC m=+39.021150259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pzd6j" (UID: "82fbbb40-345c-41b3-ac15-49360dcd7859") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:17.123900 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:17.123876    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm podName:82fbbb40-345c-41b3-ac15-49360dcd7859 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:17.623851696 +0000 UTC m=+39.021194413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pzd6j" (UID: "82fbbb40-345c-41b3-ac15-49360dcd7859") : configmap "kube-rbac-proxy" not found
Sep 11 13:49:17.124314 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.123758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgqvj\" (UniqueName: \"kubernetes.io/projected/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-api-access-jgqvj\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:17.127329 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.127296    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/82fbbb40-345c-41b3-ac15-49360dcd7859-data-volume\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:17.133754 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.133722    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgqvj\" (UniqueName: \"kubernetes.io/projected/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-api-access-jgqvj\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:17.391864 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.391830    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-vzpvz"]
Sep 11 13:49:17.409122 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.409091    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-vzpvz"]
Sep 11 13:49:17.409260 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.409215    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vzpvz"
Sep 11 13:49:17.411371 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.411345    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 11 13:49:17.411514 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.411464    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 11 13:49:17.411594 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.411510    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-v2qsc\""
Sep 11 13:49:17.476529 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.476493    2563 generic.go:356] "Generic (PLEG): container finished" podID="137f87ff-936c-417c-88f9-33c2a48d721f" containerID="988bd7f7a0c74e537c94057dc833c41138af0ec5cbcf4aea83a0f63ca7cc9044" exitCode=0
Sep 11 13:49:17.476984 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.476541    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jkd4s" event={"ID":"137f87ff-936c-417c-88f9-33c2a48d721f","Type":"ContainerDied","Data":"988bd7f7a0c74e537c94057dc833c41138af0ec5cbcf4aea83a0f63ca7cc9044"}
Sep 11 13:49:17.527212 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.527170    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt2jq\" (UniqueName: \"kubernetes.io/projected/a9a9504d-ec72-44f4-b419-21e33712d558-kube-api-access-zt2jq\") pod \"migrator-8b4d697c5-vzpvz\" (UID: \"a9a9504d-ec72-44f4-b419-21e33712d558\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vzpvz"
Sep 11 13:49:17.628146 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.628107    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:17.628146 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.628143    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt2jq\" (UniqueName: \"kubernetes.io/projected/a9a9504d-ec72-44f4-b419-21e33712d558-kube-api-access-zt2jq\") pod \"migrator-8b4d697c5-vzpvz\" (UID: \"a9a9504d-ec72-44f4-b419-21e33712d558\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vzpvz"
Sep 11 13:49:17.628395 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.628370    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:17.628554 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:17.628536    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:49:17.628554 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.628545    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:17.628698 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:17.628559    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfbdd748c-l979v: secret "image-registry-tls" not found
Sep 11 13:49:17.628698 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:17.628614    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls podName:fac2fe28-c77e-4648-b789-b18824e80a3f nodeName:}" failed. No retries permitted until 2026-09-11 13:49:21.62859836 +0000 UTC m=+43.025941088 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls") pod "image-registry-6cfbdd748c-l979v" (UID: "fac2fe28-c77e-4648-b789-b18824e80a3f") : secret "image-registry-tls" not found
Sep 11 13:49:17.628804 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:17.628755    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:49:17.628865 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:17.628804    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm podName:82fbbb40-345c-41b3-ac15-49360dcd7859 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:18.628788627 +0000 UTC m=+40.026131342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pzd6j" (UID: "82fbbb40-345c-41b3-ac15-49360dcd7859") : configmap "kube-rbac-proxy" not found
Sep 11 13:49:17.628865 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:17.628804    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:17.629001 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:17.628880    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls podName:82fbbb40-345c-41b3-ac15-49360dcd7859 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:18.628868254 +0000 UTC m=+40.026210969 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pzd6j" (UID: "82fbbb40-345c-41b3-ac15-49360dcd7859") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:17.636379 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.636353    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt2jq\" (UniqueName: \"kubernetes.io/projected/a9a9504d-ec72-44f4-b419-21e33712d558-kube-api-access-zt2jq\") pod \"migrator-8b4d697c5-vzpvz\" (UID: \"a9a9504d-ec72-44f4-b419-21e33712d558\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vzpvz"
Sep 11 13:49:17.719029 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.718948    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vzpvz"
Sep 11 13:49:17.729920 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:17.729889    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:17.730122 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:17.730055    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:49:17.730246 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:17.730135    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls podName:2054806b-ed98-4a9c-b889-0a4efda17d75 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:21.730112563 +0000 UTC m=+43.127455282 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls") pod "dns-default-rxc9q" (UID: "2054806b-ed98-4a9c-b889-0a4efda17d75") : secret "dns-default-metrics-tls" not found
Sep 11 13:49:18.463810 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:18.463662    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7q9dt_071b3477-ebf8-450b-9ca7-c2767dea40ea/dns-node-resolver/0.log"
Sep 11 13:49:18.482800 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:18.482773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jkd4s" event={"ID":"137f87ff-936c-417c-88f9-33c2a48d721f","Type":"ContainerStarted","Data":"87348947a77e14b8bee9852d7707880e47471e153128cb2fb6a7ae9721829e67"}
Sep 11 13:49:18.503600 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:18.503544    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jkd4s" podStartSLOduration=8.396725882 podStartE2EDuration="38.503520923s" totalImagesPullingTime="30.106795041s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:18.501832599 +0000 UTC m=+39.899175337" watchObservedRunningTime="2026-09-11 13:49:18.503520923 +0000 UTC m=+39.900863659"
Sep 11 13:49:18.613375 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:18.613293    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-vzpvz"]
Sep 11 13:49:18.639575 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:18.639538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:18.639746 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:18.639674    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:49:18.639746 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:18.639740    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm podName:82fbbb40-345c-41b3-ac15-49360dcd7859 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:20.639720049 +0000 UTC m=+42.037062771 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pzd6j" (UID: "82fbbb40-345c-41b3-ac15-49360dcd7859") : configmap "kube-rbac-proxy" not found
Sep 11 13:49:18.639851 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:18.639786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:18.639903 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:18.639895    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:18.639946 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:18.639942    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls podName:82fbbb40-345c-41b3-ac15-49360dcd7859 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:20.639929715 +0000 UTC m=+42.037272430 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pzd6j" (UID: "82fbbb40-345c-41b3-ac15-49360dcd7859") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:18.699364 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:49:18.699324    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9a9504d_ec72_44f4_b419_21e33712d558.slice/crio-ee314943a1fad652a78c15d456525b100ee5097b1b585c31c301da540d10c73f WatchSource:0}: Error finding container ee314943a1fad652a78c15d456525b100ee5097b1b585c31c301da540d10c73f: Status 404 returned error can't find the container with id ee314943a1fad652a78c15d456525b100ee5097b1b585c31c301da540d10c73f
Sep 11 13:49:19.258750 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.258719    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5frp2_d39e8d10-0ae8-4265-9b99-ac2ac072b1ab/node-ca/0.log"
Sep 11 13:49:19.487074 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.487013    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8cwcd" event={"ID":"a3d7f55a-5d08-4e04-a09c-73c237ab55e8","Type":"ContainerStarted","Data":"4775c5b33f0a97ba7478cfe9a28100ded14ff069f32a7971c43bcbb8cea2fce8"}
Sep 11 13:49:19.488452 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.488419    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vzpvz" event={"ID":"a9a9504d-ec72-44f4-b419-21e33712d558","Type":"ContainerStarted","Data":"ee314943a1fad652a78c15d456525b100ee5097b1b585c31c301da540d10c73f"}
Sep 11 13:49:19.502224 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.502159    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8cwcd" podStartSLOduration=3.5489714169999997 podStartE2EDuration="6.502135684s" totalImagesPullingTime="2.953164267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:19.501179937 +0000 UTC m=+40.898522675" watchObservedRunningTime="2026-09-11 13:49:19.502135684 +0000 UTC m=+40.899478422"
Sep 11 13:49:19.524526 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.524445    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-jv9zp"]
Sep 11 13:49:19.538899 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.538869    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-jv9zp"
Sep 11 13:49:19.539048 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.538964    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-jv9zp"]
Sep 11 13:49:19.542657 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.541595    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9lwrn\""
Sep 11 13:49:19.543590 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.543567    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 11 13:49:19.543870 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.543857    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 11 13:49:19.544321 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.544304    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 11 13:49:19.544525 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.544508    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 11 13:49:19.649072 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.649035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/034c553d-881b-472f-b1a3-b86deda8ec3a-signing-key\") pod \"service-ca-568869fdc5-jv9zp\" (UID: \"034c553d-881b-472f-b1a3-b86deda8ec3a\") " pod="openshift-service-ca/service-ca-568869fdc5-jv9zp"
Sep 11 13:49:19.649221 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.649081    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6456w\" (UniqueName: \"kubernetes.io/projected/034c553d-881b-472f-b1a3-b86deda8ec3a-kube-api-access-6456w\") pod \"service-ca-568869fdc5-jv9zp\" (UID: \"034c553d-881b-472f-b1a3-b86deda8ec3a\") " pod="openshift-service-ca/service-ca-568869fdc5-jv9zp"
Sep 11 13:49:19.649221 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.649177    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/034c553d-881b-472f-b1a3-b86deda8ec3a-signing-cabundle\") pod \"service-ca-568869fdc5-jv9zp\" (UID: \"034c553d-881b-472f-b1a3-b86deda8ec3a\") " pod="openshift-service-ca/service-ca-568869fdc5-jv9zp"
Sep 11 13:49:19.750719 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.750674    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/034c553d-881b-472f-b1a3-b86deda8ec3a-signing-cabundle\") pod \"service-ca-568869fdc5-jv9zp\" (UID: \"034c553d-881b-472f-b1a3-b86deda8ec3a\") " pod="openshift-service-ca/service-ca-568869fdc5-jv9zp"
Sep 11 13:49:19.750877 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.750833    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/034c553d-881b-472f-b1a3-b86deda8ec3a-signing-key\") pod \"service-ca-568869fdc5-jv9zp\" (UID: \"034c553d-881b-472f-b1a3-b86deda8ec3a\") " pod="openshift-service-ca/service-ca-568869fdc5-jv9zp"
Sep 11 13:49:19.750877 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.750860    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6456w\" (UniqueName: \"kubernetes.io/projected/034c553d-881b-472f-b1a3-b86deda8ec3a-kube-api-access-6456w\") pod \"service-ca-568869fdc5-jv9zp\" (UID: \"034c553d-881b-472f-b1a3-b86deda8ec3a\") " pod="openshift-service-ca/service-ca-568869fdc5-jv9zp"
Sep 11 13:49:19.751525 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.751496    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/034c553d-881b-472f-b1a3-b86deda8ec3a-signing-cabundle\") pod \"service-ca-568869fdc5-jv9zp\" (UID: \"034c553d-881b-472f-b1a3-b86deda8ec3a\") " pod="openshift-service-ca/service-ca-568869fdc5-jv9zp"
Sep 11 13:49:19.754808 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.754777    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/034c553d-881b-472f-b1a3-b86deda8ec3a-signing-key\") pod \"service-ca-568869fdc5-jv9zp\" (UID: \"034c553d-881b-472f-b1a3-b86deda8ec3a\") " pod="openshift-service-ca/service-ca-568869fdc5-jv9zp"
Sep 11 13:49:19.759107 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.759087    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6456w\" (UniqueName: \"kubernetes.io/projected/034c553d-881b-472f-b1a3-b86deda8ec3a-kube-api-access-6456w\") pod \"service-ca-568869fdc5-jv9zp\" (UID: \"034c553d-881b-472f-b1a3-b86deda8ec3a\") " pod="openshift-service-ca/service-ca-568869fdc5-jv9zp"
Sep 11 13:49:19.852041 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:19.851999    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-jv9zp"
Sep 11 13:49:20.219111 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:20.219032    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-jv9zp"]
Sep 11 13:49:20.239040 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:49:20.239008    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034c553d_881b_472f_b1a3_b86deda8ec3a.slice/crio-b9b9f5b3c7cf0cebdf76991a961b8df029961178b1b788d13526037cb96608dd WatchSource:0}: Error finding container b9b9f5b3c7cf0cebdf76991a961b8df029961178b1b788d13526037cb96608dd: Status 404 returned error can't find the container with id b9b9f5b3c7cf0cebdf76991a961b8df029961178b1b788d13526037cb96608dd
Sep 11 13:49:20.259805 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:20.259778    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8cwcd_a3d7f55a-5d08-4e04-a09c-73c237ab55e8/serve-healthcheck-canary/0.log"
Sep 11 13:49:20.492224 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:20.492135    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vzpvz" event={"ID":"a9a9504d-ec72-44f4-b419-21e33712d558","Type":"ContainerStarted","Data":"2064e9584663f8b0eff9c66fe381a4664c9c1e569c4e8ffb6519b005ec9bef25"}
Sep 11 13:49:20.493311 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:20.493287    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-jv9zp" event={"ID":"034c553d-881b-472f-b1a3-b86deda8ec3a","Type":"ContainerStarted","Data":"b9b9f5b3c7cf0cebdf76991a961b8df029961178b1b788d13526037cb96608dd"}
Sep 11 13:49:20.658245 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:20.658208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:20.658410 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:20.658338    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:49:20.658410 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:20.658399    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:20.658504 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:20.658414    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm podName:82fbbb40-345c-41b3-ac15-49360dcd7859 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:24.658394824 +0000 UTC m=+46.055737570 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pzd6j" (UID: "82fbbb40-345c-41b3-ac15-49360dcd7859") : configmap "kube-rbac-proxy" not found
Sep 11 13:49:20.658504 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:20.658477    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:20.658613 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:20.658507    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls podName:82fbbb40-345c-41b3-ac15-49360dcd7859 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:24.658498909 +0000 UTC m=+46.055841624 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pzd6j" (UID: "82fbbb40-345c-41b3-ac15-49360dcd7859") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:21.498254 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:21.498218    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vzpvz" event={"ID":"a9a9504d-ec72-44f4-b419-21e33712d558","Type":"ContainerStarted","Data":"715bf0547f3afcf9cfc27b1515d95edef9a2ff763151e3e9d12dea992b30856c"}
Sep 11 13:49:21.513826 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:21.513777    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-vzpvz" podStartSLOduration=2.9636693640000003 podStartE2EDuration="4.513760514s" totalImagesPullingTime="1.55009115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:21.512674277 +0000 UTC m=+42.910017011" watchObservedRunningTime="2026-09-11 13:49:21.513760514 +0000 UTC m=+42.911103288"
Sep 11 13:49:21.668573 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:21.668539    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:21.668763 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:21.668730    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:49:21.668763 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:21.668752    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfbdd748c-l979v: secret "image-registry-tls" not found
Sep 11 13:49:21.668829 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:21.668819    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls podName:fac2fe28-c77e-4648-b789-b18824e80a3f nodeName:}" failed. No retries permitted until 2026-09-11 13:49:29.668804034 +0000 UTC m=+51.066146750 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls") pod "image-registry-6cfbdd748c-l979v" (UID: "fac2fe28-c77e-4648-b789-b18824e80a3f") : secret "image-registry-tls" not found
Sep 11 13:49:21.769973 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:21.769876    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:21.770146 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:21.770028    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:49:21.770146 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:21.770107    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls podName:2054806b-ed98-4a9c-b889-0a4efda17d75 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:29.770084876 +0000 UTC m=+51.167427592 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls") pod "dns-default-rxc9q" (UID: "2054806b-ed98-4a9c-b889-0a4efda17d75") : secret "dns-default-metrics-tls" not found
Sep 11 13:49:23.504075 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:23.504039    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-jv9zp" event={"ID":"034c553d-881b-472f-b1a3-b86deda8ec3a","Type":"ContainerStarted","Data":"d9393e505b9ac84110343366b507e47489dec2bf6ae5c460abad6d6487468edf"}
Sep 11 13:49:23.520529 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:23.520474    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-jv9zp" podStartSLOduration=2.173171864 podStartE2EDuration="4.520457755s" totalImagesPullingTime="2.347285891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:23.519917877 +0000 UTC m=+44.917260616" watchObservedRunningTime="2026-09-11 13:49:23.520457755 +0000 UTC m=+44.917800492"
Sep 11 13:49:24.698284 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:24.698240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:24.698872 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:24.698410    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:24.698997 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:24.698921    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls podName:82fbbb40-345c-41b3-ac15-49360dcd7859 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:32.69889774 +0000 UTC m=+54.096240462 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pzd6j" (UID: "82fbbb40-345c-41b3-ac15-49360dcd7859") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:24.698997 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:24.698965    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:24.699121 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:24.699056    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:49:24.699121 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:24.699090    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm podName:82fbbb40-345c-41b3-ac15-49360dcd7859 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:32.699079641 +0000 UTC m=+54.096422358 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pzd6j" (UID: "82fbbb40-345c-41b3-ac15-49360dcd7859") : configmap "kube-rbac-proxy" not found
Sep 11 13:49:29.737918 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:29.737881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:29.740582 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:29.740553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls\") pod \"image-registry-6cfbdd748c-l979v\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") " pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:29.838947 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:29.838893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:29.839391 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:29.839363    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:29.841451 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:29.841427    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2054806b-ed98-4a9c-b889-0a4efda17d75-metrics-tls\") pod \"dns-default-rxc9q\" (UID: \"2054806b-ed98-4a9c-b889-0a4efda17d75\") " pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:29.851525 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:29.851505    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:30.114069 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:30.114037    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cfbdd748c-l979v"]
Sep 11 13:49:30.117222 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:49:30.117197    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfac2fe28_c77e_4648_b789_b18824e80a3f.slice/crio-3252793c23d35f114e9d57cc543bfa933a09367951bf1286f5662aae4902da35 WatchSource:0}: Error finding container 3252793c23d35f114e9d57cc543bfa933a09367951bf1286f5662aae4902da35: Status 404 returned error can't find the container with id 3252793c23d35f114e9d57cc543bfa933a09367951bf1286f5662aae4902da35
Sep 11 13:49:30.129212 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:30.129188    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rxc9q"]
Sep 11 13:49:30.132811 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:49:30.132779    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2054806b_ed98_4a9c_b889_0a4efda17d75.slice/crio-258e8cc7a25063a66b2610cc65a179d152285d6b5758adc930e4962d7f04845f WatchSource:0}: Error finding container 258e8cc7a25063a66b2610cc65a179d152285d6b5758adc930e4962d7f04845f: Status 404 returned error can't find the container with id 258e8cc7a25063a66b2610cc65a179d152285d6b5758adc930e4962d7f04845f
Sep 11 13:49:30.523592 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:30.523500    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cfbdd748c-l979v" event={"ID":"fac2fe28-c77e-4648-b789-b18824e80a3f","Type":"ContainerStarted","Data":"7cfec4c3758da380b8c10b55f35ce3edd89ce9f6846e05d864227bd51159aa0f"}
Sep 11 13:49:30.523592 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:30.523561    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cfbdd748c-l979v" event={"ID":"fac2fe28-c77e-4648-b789-b18824e80a3f","Type":"ContainerStarted","Data":"3252793c23d35f114e9d57cc543bfa933a09367951bf1286f5662aae4902da35"}
Sep 11 13:49:30.523592 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:30.523587    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:30.524888 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:30.524864    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rxc9q" event={"ID":"2054806b-ed98-4a9c-b889-0a4efda17d75","Type":"ContainerStarted","Data":"258e8cc7a25063a66b2610cc65a179d152285d6b5758adc930e4962d7f04845f"}
Sep 11 13:49:30.545126 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:30.545071    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cfbdd748c-l979v" podStartSLOduration=22.545050745 podStartE2EDuration="22.545050745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:30.544136653 +0000 UTC m=+51.941479395" watchObservedRunningTime="2026-09-11 13:49:30.545050745 +0000 UTC m=+51.942393483"
Sep 11 13:49:32.533503 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:32.533407    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rxc9q" event={"ID":"2054806b-ed98-4a9c-b889-0a4efda17d75","Type":"ContainerStarted","Data":"ee64f395f6439196261391a1b2f63e6c92ec3b9d59d14b6496569a9144a853bc"}
Sep 11 13:49:32.533503 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:32.533468    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rxc9q" event={"ID":"2054806b-ed98-4a9c-b889-0a4efda17d75","Type":"ContainerStarted","Data":"277ddf9d1d0d6ba501af766fa500f5753bdc0bdea435bc599d7e840efdb03a4f"}
Sep 11 13:49:32.534001 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:32.533537    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:32.550203 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:32.550145    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rxc9q" podStartSLOduration=17.543389748 podStartE2EDuration="19.550124862s" totalImagesPullingTime="2.006735114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:32.548555413 +0000 UTC m=+53.945898150" watchObservedRunningTime="2026-09-11 13:49:32.550124862 +0000 UTC m=+53.947467605"
Sep 11 13:49:32.762360 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:32.762323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:32.762524 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:32.762376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:32.762524 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:32.762473    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:32.762595 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:32.762542    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls podName:82fbbb40-345c-41b3-ac15-49360dcd7859 nodeName:}" failed. No retries permitted until 2026-09-11 13:49:48.762523715 +0000 UTC m=+70.159866430 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pzd6j" (UID: "82fbbb40-345c-41b3-ac15-49360dcd7859") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:49:32.763500 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:32.763476    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82fbbb40-345c-41b3-ac15-49360dcd7859-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:39.455552 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:39.455518    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c4hqn"
Sep 11 13:49:40.858473 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.858443    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7"]
Sep 11 13:49:40.861351 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.861331    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7"
Sep 11 13:49:40.862868 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.862841    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"]
Sep 11 13:49:40.864085 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.864062    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 13:49:40.864521 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.864495    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 13:49:40.864786 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.864765    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 13:49:40.865055 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.865035    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 11 13:49:40.865155 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.865134    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-g2k7f\""
Sep 11 13:49:40.868314 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.868286    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"]
Sep 11 13:49:40.868473 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.868437    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"
Sep 11 13:49:40.870494 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.870477    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:40.871447 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.871426    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 11 13:49:40.872334 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.872316    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 11 13:49:40.872422 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.872366    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7"]
Sep 11 13:49:40.872705 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.872689    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 11 13:49:40.872791 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.872761    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 11 13:49:40.872849 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.872798    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 11 13:49:40.877931 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.877911    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cfbdd748c-l979v"]
Sep 11 13:49:40.878639 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.878606    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"]
Sep 11 13:49:40.883650 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.883610    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"]
Sep 11 13:49:40.922383 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.922347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c182b9dd-7c7f-42a5-8ec1-288260f1e590-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7\" (UID: \"c182b9dd-7c7f-42a5-8ec1-288260f1e590\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7"
Sep 11 13:49:40.922383 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.922389    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bfd73cf4-399b-4035-b4d7-72311def9593-klusterlet-config\") pod \"klusterlet-addon-workmgr-67fbbc66c4-rxmpj\" (UID: \"bfd73cf4-399b-4035-b4d7-72311def9593\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"
Sep 11 13:49:40.922603 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.922503    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/cd6f62b6-2848-4fd0-a083-c4780125226e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:40.922603 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.922553    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/cd6f62b6-2848-4fd0-a083-c4780125226e-ca\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:40.922603 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.922576    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd6f62b6-2848-4fd0-a083-c4780125226e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:40.922603 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.922600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cd6f62b6-2848-4fd0-a083-c4780125226e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:40.922768 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.922648    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7qv5\" (UniqueName: \"kubernetes.io/projected/cd6f62b6-2848-4fd0-a083-c4780125226e-kube-api-access-z7qv5\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:40.922768 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.922683    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg96m\" (UniqueName: \"kubernetes.io/projected/bfd73cf4-399b-4035-b4d7-72311def9593-kube-api-access-cg96m\") pod \"klusterlet-addon-workmgr-67fbbc66c4-rxmpj\" (UID: \"bfd73cf4-399b-4035-b4d7-72311def9593\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"
Sep 11 13:49:40.922768 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.922699    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfd73cf4-399b-4035-b4d7-72311def9593-tmp\") pod \"klusterlet-addon-workmgr-67fbbc66c4-rxmpj\" (UID: \"bfd73cf4-399b-4035-b4d7-72311def9593\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"
Sep 11 13:49:40.922768 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.922743    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g9sm\" (UniqueName: \"kubernetes.io/projected/c182b9dd-7c7f-42a5-8ec1-288260f1e590-kube-api-access-4g9sm\") pod \"managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7\" (UID: \"c182b9dd-7c7f-42a5-8ec1-288260f1e590\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7"
Sep 11 13:49:40.922768 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.922758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/cd6f62b6-2848-4fd0-a083-c4780125226e-hub\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:40.963436 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.963405    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f77897478-vbwcm"]
Sep 11 13:49:40.965338 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.965323    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:40.984994 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:40.984970    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f77897478-vbwcm"]
Sep 11 13:49:41.023985 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.023948    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/cd6f62b6-2848-4fd0-a083-c4780125226e-ca\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:41.023985 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.023981    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd6f62b6-2848-4fd0-a083-c4780125226e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:41.024250 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024009    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-registry-tls\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.024250 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cd6f62b6-2848-4fd0-a083-c4780125226e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:41.024250 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024047    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7qv5\" (UniqueName: \"kubernetes.io/projected/cd6f62b6-2848-4fd0-a083-c4780125226e-kube-api-access-z7qv5\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:41.024250 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-registry-certificates\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.024250 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg96m\" (UniqueName: \"kubernetes.io/projected/bfd73cf4-399b-4035-b4d7-72311def9593-kube-api-access-cg96m\") pod \"klusterlet-addon-workmgr-67fbbc66c4-rxmpj\" (UID: \"bfd73cf4-399b-4035-b4d7-72311def9593\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"
Sep 11 13:49:41.024250 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024176    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-ca-trust-extracted\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.024569 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfd73cf4-399b-4035-b4d7-72311def9593-tmp\") pod \"klusterlet-addon-workmgr-67fbbc66c4-rxmpj\" (UID: \"bfd73cf4-399b-4035-b4d7-72311def9593\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"
Sep 11 13:49:41.024569 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024306    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h555m\" (UniqueName: \"kubernetes.io/projected/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-kube-api-access-h555m\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.024569 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024342    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-image-registry-private-configuration\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.024569 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g9sm\" (UniqueName: \"kubernetes.io/projected/c182b9dd-7c7f-42a5-8ec1-288260f1e590-kube-api-access-4g9sm\") pod \"managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7\" (UID: \"c182b9dd-7c7f-42a5-8ec1-288260f1e590\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7"
Sep 11 13:49:41.024569 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024400    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/cd6f62b6-2848-4fd0-a083-c4780125226e-hub\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:41.024569 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024439    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c182b9dd-7c7f-42a5-8ec1-288260f1e590-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7\" (UID: \"c182b9dd-7c7f-42a5-8ec1-288260f1e590\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7"
Sep 11 13:49:41.024569 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bfd73cf4-399b-4035-b4d7-72311def9593-klusterlet-config\") pod \"klusterlet-addon-workmgr-67fbbc66c4-rxmpj\" (UID: \"bfd73cf4-399b-4035-b4d7-72311def9593\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"
Sep 11 13:49:41.024569 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-trusted-ca\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.025006 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024579    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-installation-pull-secrets\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.025006 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024607    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-bound-sa-token\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.025006 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfd73cf4-399b-4035-b4d7-72311def9593-tmp\") pod \"klusterlet-addon-workmgr-67fbbc66c4-rxmpj\" (UID: \"bfd73cf4-399b-4035-b4d7-72311def9593\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"
Sep 11 13:49:41.025006 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/cd6f62b6-2848-4fd0-a083-c4780125226e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:41.025006 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.024849    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd6f62b6-2848-4fd0-a083-c4780125226e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:41.026926 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.026897    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/cd6f62b6-2848-4fd0-a083-c4780125226e-ca\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:41.027313 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.027288    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c182b9dd-7c7f-42a5-8ec1-288260f1e590-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7\" (UID: \"c182b9dd-7c7f-42a5-8ec1-288260f1e590\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7"
Sep 11 13:49:41.027454 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.027433    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/cd6f62b6-2848-4fd0-a083-c4780125226e-hub\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:41.027517 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.027482    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cd6f62b6-2848-4fd0-a083-c4780125226e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:41.027616 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.027600    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/cd6f62b6-2848-4fd0-a083-c4780125226e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:41.027909 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.027889    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bfd73cf4-399b-4035-b4d7-72311def9593-klusterlet-config\") pod \"klusterlet-addon-workmgr-67fbbc66c4-rxmpj\" (UID: \"bfd73cf4-399b-4035-b4d7-72311def9593\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"
Sep 11 13:49:41.043840 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.043809    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7qv5\" (UniqueName: \"kubernetes.io/projected/cd6f62b6-2848-4fd0-a083-c4780125226e-kube-api-access-z7qv5\") pod \"cluster-proxy-proxy-agent-694bf6d5c-xt9pc\" (UID: \"cd6f62b6-2848-4fd0-a083-c4780125226e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:41.045151 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.045122    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg96m\" (UniqueName: \"kubernetes.io/projected/bfd73cf4-399b-4035-b4d7-72311def9593-kube-api-access-cg96m\") pod \"klusterlet-addon-workmgr-67fbbc66c4-rxmpj\" (UID: \"bfd73cf4-399b-4035-b4d7-72311def9593\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"
Sep 11 13:49:41.045787 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.045762    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g9sm\" (UniqueName: \"kubernetes.io/projected/c182b9dd-7c7f-42a5-8ec1-288260f1e590-kube-api-access-4g9sm\") pod \"managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7\" (UID: \"c182b9dd-7c7f-42a5-8ec1-288260f1e590\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7"
Sep 11 13:49:41.125521 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.125437    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h555m\" (UniqueName: \"kubernetes.io/projected/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-kube-api-access-h555m\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.125521 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.125476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-image-registry-private-configuration\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.125521 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.125513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-trusted-ca\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.125813 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.125528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-installation-pull-secrets\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.125813 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.125545    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-bound-sa-token\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.125813 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.125735    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-registry-tls\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.125813 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.125785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-registry-certificates\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.126053 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.125824    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-ca-trust-extracted\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.126243 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.126222    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-ca-trust-extracted\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.126820 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.126795    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-registry-certificates\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.126820 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.126798    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-trusted-ca\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.128183 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.128159    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-installation-pull-secrets\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.128389 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.128367    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-registry-tls\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.128456 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.128405    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-image-registry-private-configuration\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.138217 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.138189    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-bound-sa-token\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.138806 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.138786    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h555m\" (UniqueName: \"kubernetes.io/projected/6a67eff1-dc1f-49ed-b3c4-24ab86713afc-kube-api-access-h555m\") pod \"image-registry-7f77897478-vbwcm\" (UID: \"6a67eff1-dc1f-49ed-b3c4-24ab86713afc\") " pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.181703 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.181665    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7"
Sep 11 13:49:41.190388 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.190359    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"
Sep 11 13:49:41.196207 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.196181    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"
Sep 11 13:49:41.287059 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.287027    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:41.457185 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.457114    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7"]
Sep 11 13:49:41.460300 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:49:41.460216    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc182b9dd_7c7f_42a5_8ec1_288260f1e590.slice/crio-bdf0b3065e69319cf8c692791237dffe87bf89f7bebe52b42fa85f8156aace9f WatchSource:0}: Error finding container bdf0b3065e69319cf8c692791237dffe87bf89f7bebe52b42fa85f8156aace9f: Status 404 returned error can't find the container with id bdf0b3065e69319cf8c692791237dffe87bf89f7bebe52b42fa85f8156aace9f
Sep 11 13:49:41.461005 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.460860    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"]
Sep 11 13:49:41.463871 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:49:41.463849    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd73cf4_399b_4035_b4d7_72311def9593.slice/crio-f92930bd575fccd72925fbe5a34611f5e17db19772c712994fe65fd49cbaa7a0 WatchSource:0}: Error finding container f92930bd575fccd72925fbe5a34611f5e17db19772c712994fe65fd49cbaa7a0: Status 404 returned error can't find the container with id f92930bd575fccd72925fbe5a34611f5e17db19772c712994fe65fd49cbaa7a0
Sep 11 13:49:41.483129 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.483109    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc"]
Sep 11 13:49:41.485250 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:49:41.485228    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd6f62b6_2848_4fd0_a083_c4780125226e.slice/crio-ae42ae63648bce103c21769b5f457493e183a07164ba26452dba45bc1ef425f6 WatchSource:0}: Error finding container ae42ae63648bce103c21769b5f457493e183a07164ba26452dba45bc1ef425f6: Status 404 returned error can't find the container with id ae42ae63648bce103c21769b5f457493e183a07164ba26452dba45bc1ef425f6
Sep 11 13:49:41.559436 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.559401    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f77897478-vbwcm"]
Sep 11 13:49:41.562507 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:49:41.562482    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a67eff1_dc1f_49ed_b3c4_24ab86713afc.slice/crio-37da08d35e69d409d17997ca3cba1e8fe7de27bccb76c4f523373ba020ac703f WatchSource:0}: Error finding container 37da08d35e69d409d17997ca3cba1e8fe7de27bccb76c4f523373ba020ac703f: Status 404 returned error can't find the container with id 37da08d35e69d409d17997ca3cba1e8fe7de27bccb76c4f523373ba020ac703f
Sep 11 13:49:41.563359 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.563324    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj" event={"ID":"bfd73cf4-399b-4035-b4d7-72311def9593","Type":"ContainerStarted","Data":"f92930bd575fccd72925fbe5a34611f5e17db19772c712994fe65fd49cbaa7a0"}
Sep 11 13:49:41.564738 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.564710    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc" event={"ID":"cd6f62b6-2848-4fd0-a083-c4780125226e","Type":"ContainerStarted","Data":"ae42ae63648bce103c21769b5f457493e183a07164ba26452dba45bc1ef425f6"}
Sep 11 13:49:41.565864 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:41.565844    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7" event={"ID":"c182b9dd-7c7f-42a5-8ec1-288260f1e590","Type":"ContainerStarted","Data":"bdf0b3065e69319cf8c692791237dffe87bf89f7bebe52b42fa85f8156aace9f"}
Sep 11 13:49:42.540658 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:42.540611    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rxc9q"
Sep 11 13:49:42.575727 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:42.575691    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f77897478-vbwcm" event={"ID":"6a67eff1-dc1f-49ed-b3c4-24ab86713afc","Type":"ContainerStarted","Data":"2891ea6a98afc2751c7b12126c2c8ebd546c56daea2c4bcacc4735e0eb8c0cf8"}
Sep 11 13:49:42.575902 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:42.575749    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f77897478-vbwcm" event={"ID":"6a67eff1-dc1f-49ed-b3c4-24ab86713afc","Type":"ContainerStarted","Data":"37da08d35e69d409d17997ca3cba1e8fe7de27bccb76c4f523373ba020ac703f"}
Sep 11 13:49:42.576451 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:42.576427    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:49:42.599925 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:42.599850    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f77897478-vbwcm" podStartSLOduration=2.599827824 podStartE2EDuration="2.599827824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:42.598331195 +0000 UTC m=+63.995673933" watchObservedRunningTime="2026-09-11 13:49:42.599827824 +0000 UTC m=+63.997170566"
Sep 11 13:49:44.964226 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:44.964183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs\") pod \"network-metrics-daemon-59892\" (UID: \"47db31ca-5cef-4c37-9da5-027f3df08815\") " pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:49:44.964710 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:44.964238    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:49:44.966335 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:44.966304    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:49:44.966472 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:44.966368    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:49:44.977938 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:44.977914    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47db31ca-5cef-4c37-9da5-027f3df08815-metrics-certs\") pod \"network-metrics-daemon-59892\" (UID: \"47db31ca-5cef-4c37-9da5-027f3df08815\") " pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:49:44.978114 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:44.978081    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c293f3c-09e5-4f86-991b-1613158f19ee-original-pull-secret\") pod \"global-pull-secret-syncer-hfjt6\" (UID: \"2c293f3c-09e5-4f86-991b-1613158f19ee\") " pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:49:45.065387 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:45.065351    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x97z\" (UniqueName: \"kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z\") pod \"network-check-target-lrb9b\" (UID: \"98c33f0c-094f-490f-af00-24b762506101\") " pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:49:45.067366 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:45.067340    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:49:45.078058 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:45.078034    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:49:45.088658 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:45.088616    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x97z\" (UniqueName: \"kubernetes.io/projected/98c33f0c-094f-490f-af00-24b762506101-kube-api-access-7x97z\") pod \"network-check-target-lrb9b\" (UID: \"98c33f0c-094f-490f-af00-24b762506101\") " pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:49:45.139125 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:45.139094    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hfjt6"
Sep 11 13:49:45.148014 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:45.147990    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mlc2t\""
Sep 11 13:49:45.153693 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:45.153671    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6pjch\""
Sep 11 13:49:45.156232 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:45.156213    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:49:45.163032 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:45.163013    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-59892"
Sep 11 13:49:45.589398 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:45.589364    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc" event={"ID":"cd6f62b6-2848-4fd0-a083-c4780125226e","Type":"ContainerStarted","Data":"cb673b7e19764a433d96bf1794451b760300c5359a5634b6be4532c6b470c276"}
Sep 11 13:49:45.591574 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:45.591546    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7" event={"ID":"c182b9dd-7c7f-42a5-8ec1-288260f1e590","Type":"ContainerStarted","Data":"b9e62ea99c7c9f2bc24d6e09a70644cd099793dd26aeb5902f9167e26a98500d"}
Sep 11 13:49:45.608205 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:45.608108    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bfcdfb68d-mdvb7" podStartSLOduration=1.6281254870000001 podStartE2EDuration="5.608091835s" totalImagesPullingTime="3.979966348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:45.607188755 +0000 UTC m=+67.004531493" watchObservedRunningTime="2026-09-11 13:49:45.608091835 +0000 UTC m=+67.005434572"
Sep 11 13:49:45.723852 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:45.723780    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hfjt6"]
Sep 11 13:49:45.727067 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:45.727036    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-59892"]
Sep 11 13:49:45.727380 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:49:45.727113    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c293f3c_09e5_4f86_991b_1613158f19ee.slice/crio-eddfe8c77fd47102e1213ea9ea6ff4dba76c12f362e2e2448d741823403c28a4 WatchSource:0}: Error finding container eddfe8c77fd47102e1213ea9ea6ff4dba76c12f362e2e2448d741823403c28a4: Status 404 returned error can't find the container with id eddfe8c77fd47102e1213ea9ea6ff4dba76c12f362e2e2448d741823403c28a4
Sep 11 13:49:45.729537 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:49:45.729511    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47db31ca_5cef_4c37_9da5_027f3df08815.slice/crio-c818c7f14a145e66b1c7d3f0675dc2691e930baceffaeb159d82780a5d28109d WatchSource:0}: Error finding container c818c7f14a145e66b1c7d3f0675dc2691e930baceffaeb159d82780a5d28109d: Status 404 returned error can't find the container with id c818c7f14a145e66b1c7d3f0675dc2691e930baceffaeb159d82780a5d28109d
Sep 11 13:49:45.739249 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:45.739225    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lrb9b"]
Sep 11 13:49:45.742222 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:49:45.742199    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c33f0c_094f_490f_af00_24b762506101.slice/crio-9f2b223e68c252b537d4aa956aab70e84a734cec14bd11d73161350007f3d464 WatchSource:0}: Error finding container 9f2b223e68c252b537d4aa956aab70e84a734cec14bd11d73161350007f3d464: Status 404 returned error can't find the container with id 9f2b223e68c252b537d4aa956aab70e84a734cec14bd11d73161350007f3d464
Sep 11 13:49:46.606678 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:46.606055    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc" event={"ID":"cd6f62b6-2848-4fd0-a083-c4780125226e","Type":"ContainerStarted","Data":"f129610b999ef71bb679183ecbbbb98325f33539029d5c76ec74db54b9426c0b"}
Sep 11 13:49:46.606678 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:46.606111    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc" event={"ID":"cd6f62b6-2848-4fd0-a083-c4780125226e","Type":"ContainerStarted","Data":"a7d9a067c345528dff53430565f131e7ff36a119c19fa25a96231aa49d9108d2"}
Sep 11 13:49:46.608429 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:46.608399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lrb9b" event={"ID":"98c33f0c-094f-490f-af00-24b762506101","Type":"ContainerStarted","Data":"9f2b223e68c252b537d4aa956aab70e84a734cec14bd11d73161350007f3d464"}
Sep 11 13:49:46.612615 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:46.612587    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hfjt6" event={"ID":"2c293f3c-09e5-4f86-991b-1613158f19ee","Type":"ContainerStarted","Data":"eddfe8c77fd47102e1213ea9ea6ff4dba76c12f362e2e2448d741823403c28a4"}
Sep 11 13:49:46.615179 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:46.615115    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-59892" event={"ID":"47db31ca-5cef-4c37-9da5-027f3df08815","Type":"ContainerStarted","Data":"c818c7f14a145e66b1c7d3f0675dc2691e930baceffaeb159d82780a5d28109d"}
Sep 11 13:49:46.624451 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:46.624400    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-694bf6d5c-xt9pc" podStartSLOduration=2.651133737 podStartE2EDuration="6.624382473s" totalImagesPullingTime="3.973248736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:46.623874359 +0000 UTC m=+68.021217099" watchObservedRunningTime="2026-09-11 13:49:46.624382473 +0000 UTC m=+68.021725227"
Sep 11 13:49:48.796371 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:48.796317    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:48.799195 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:48.799167    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82fbbb40-345c-41b3-ac15-49360dcd7859-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pzd6j\" (UID: \"82fbbb40-345c-41b3-ac15-49360dcd7859\") " pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:49.025107 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:49.025075    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-klbks\""
Sep 11 13:49:49.033783 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:49.033753    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pzd6j"
Sep 11 13:49:50.884050 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:50.884015    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:49:52.408111 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.408027    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pzd6j"]
Sep 11 13:49:52.411356 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:49:52.411322    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82fbbb40_345c_41b3_ac15_49360dcd7859.slice/crio-a6648d2591177e95d4239eecf24c91e78e5ab202028fc51bfeb3333f0fd87972 WatchSource:0}: Error finding container a6648d2591177e95d4239eecf24c91e78e5ab202028fc51bfeb3333f0fd87972: Status 404 returned error can't find the container with id a6648d2591177e95d4239eecf24c91e78e5ab202028fc51bfeb3333f0fd87972
Sep 11 13:49:52.639547 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.639508    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj" event={"ID":"bfd73cf4-399b-4035-b4d7-72311def9593","Type":"ContainerStarted","Data":"218543b6e5306c5fddef817d2348b99f10fdb2e30cbef895ac3840feaefebc9b"}
Sep 11 13:49:52.639850 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.639675    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"
Sep 11 13:49:52.641539 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.641507    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lrb9b" event={"ID":"98c33f0c-094f-490f-af00-24b762506101","Type":"ContainerStarted","Data":"1ac98d7a806192220554b362eec8e67bde4d4eaa69e25bb9d6e11f85c0c7f58d"}
Sep 11 13:49:52.641716 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.641574    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:49:52.641815 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.641800    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj"
Sep 11 13:49:52.643177 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.643150    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hfjt6" event={"ID":"2c293f3c-09e5-4f86-991b-1613158f19ee","Type":"ContainerStarted","Data":"a2b092b721a775e7c30529733e5ed6cfcfa5b1a99105673a45993ca6eee1b696"}
Sep 11 13:49:52.645134 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.645108    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-59892" event={"ID":"47db31ca-5cef-4c37-9da5-027f3df08815","Type":"ContainerStarted","Data":"37be9746df31c5d033795856bded7d495ddcecee2e0d09a37b29040550d2f91e"}
Sep 11 13:49:52.645254 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.645156    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-59892" event={"ID":"47db31ca-5cef-4c37-9da5-027f3df08815","Type":"ContainerStarted","Data":"3db3213aa9169db4645c442129baf67ef5bcde24cbadaa0854fce5eb0ed16869"}
Sep 11 13:49:52.646667 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.646646    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pzd6j" event={"ID":"82fbbb40-345c-41b3-ac15-49360dcd7859","Type":"ContainerStarted","Data":"65a9ad74937ecb9bc40182f9e693698b1799d8a6b5e6f3b8882b09375ec20617"}
Sep 11 13:49:52.646757 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.646684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pzd6j" event={"ID":"82fbbb40-345c-41b3-ac15-49360dcd7859","Type":"ContainerStarted","Data":"a6648d2591177e95d4239eecf24c91e78e5ab202028fc51bfeb3333f0fd87972"}
Sep 11 13:49:52.661129 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.661042    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67fbbc66c4-rxmpj" podStartSLOduration=1.956857136 podStartE2EDuration="12.661029232s" totalImagesPullingTime="10.704172096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:52.659986578 +0000 UTC m=+74.057329316" watchObservedRunningTime="2026-09-11 13:49:52.661029232 +0000 UTC m=+74.058371969"
Sep 11 13:49:52.676198 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.676146    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-59892" podStartSLOduration=67.273855313 podStartE2EDuration="1m13.676129682s" totalImagesPullingTime="6.402274369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:52.676066332 +0000 UTC m=+74.073409070" watchObservedRunningTime="2026-09-11 13:49:52.676129682 +0000 UTC m=+74.073472421"
Sep 11 13:49:52.716580 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.716531    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lrb9b" podStartSLOduration=67.319466829 podStartE2EDuration="1m13.716512229s" totalImagesPullingTime="6.3970454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:52.71622352 +0000 UTC m=+74.113566258" watchObservedRunningTime="2026-09-11 13:49:52.716512229 +0000 UTC m=+74.113854966"
Sep 11 13:49:52.739209 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.739162    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hfjt6" podStartSLOduration=66.273360517 podStartE2EDuration="1m12.739148467s" totalImagesPullingTime="6.46578795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:52.738108644 +0000 UTC m=+74.135451382" watchObservedRunningTime="2026-09-11 13:49:52.739148467 +0000 UTC m=+74.136491208"
Sep 11 13:49:52.917661 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.917562    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-th29l"]
Sep 11 13:49:52.921396 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.919868    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:52.923299 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.923270    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 13:49:52.923299 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.923515    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 13:49:52.924027 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.924008    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 13:49:52.924396 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.924366    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 13:49:52.925340 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.924409    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 13:49:52.925340 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.924465    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gfth2\""
Sep 11 13:49:52.925340 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:52.924493    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 13:49:53.032616 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.032578    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-textfile\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.032863 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.032618    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-486mm\" (UniqueName: \"kubernetes.io/projected/61776a5c-0574-4877-af8e-f94a5309536f-kube-api-access-486mm\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.032863 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.032725    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61776a5c-0574-4877-af8e-f94a5309536f-sys\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.032863 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.032764    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-accelerators-collector-config\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.032863 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.032802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61776a5c-0574-4877-af8e-f94a5309536f-metrics-client-ca\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.032863 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.032839    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/61776a5c-0574-4877-af8e-f94a5309536f-root\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.032863 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.032863    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-wtmp\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.033217 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.032925    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-tls\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.033217 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.032994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.134147 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.134004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61776a5c-0574-4877-af8e-f94a5309536f-sys\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.134147 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.134039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-accelerators-collector-config\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.134147 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.134070    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61776a5c-0574-4877-af8e-f94a5309536f-metrics-client-ca\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.134147 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.134101    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/61776a5c-0574-4877-af8e-f94a5309536f-root\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.134147 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.134124    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-wtmp\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.134469 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.134150    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-tls\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.134469 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.134188    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.134469 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.134238    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-textfile\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.134469 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.134262    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-486mm\" (UniqueName: \"kubernetes.io/projected/61776a5c-0574-4877-af8e-f94a5309536f-kube-api-access-486mm\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.134706 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.134576    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61776a5c-0574-4877-af8e-f94a5309536f-sys\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.134831 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.134809    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-wtmp\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.135976 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.135333    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61776a5c-0574-4877-af8e-f94a5309536f-metrics-client-ca\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.135976 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.135398    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/61776a5c-0574-4877-af8e-f94a5309536f-root\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.135976 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:53.135784    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 11 13:49:53.135976 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:49:53.135847    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-tls podName:61776a5c-0574-4877-af8e-f94a5309536f nodeName:}" failed. No retries permitted until 2026-09-11 13:49:53.635828408 +0000 UTC m=+75.033171131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-tls") pod "node-exporter-th29l" (UID: "61776a5c-0574-4877-af8e-f94a5309536f") : secret "node-exporter-tls" not found
Sep 11 13:49:53.135976 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.135937    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-textfile\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.136662 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.136641    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-accelerators-collector-config\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.139131 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.139084    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.146580 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.146475    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-486mm\" (UniqueName: \"kubernetes.io/projected/61776a5c-0574-4877-af8e-f94a5309536f-kube-api-access-486mm\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.638696 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.638653    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-tls\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.642080 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.641962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61776a5c-0574-4877-af8e-f94a5309536f-node-exporter-tls\") pod \"node-exporter-th29l\" (UID: \"61776a5c-0574-4877-af8e-f94a5309536f\") " pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:53.655354 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.655313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pzd6j" event={"ID":"82fbbb40-345c-41b3-ac15-49360dcd7859","Type":"ContainerStarted","Data":"f4730423ce0935adfdf179d4ed4cae6ff2ec76e1941ed54d753ec1f4edfc46bf"}
Sep 11 13:49:53.835096 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:53.834936    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-th29l"
Sep 11 13:49:54.661645 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:54.661544    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pzd6j" event={"ID":"82fbbb40-345c-41b3-ac15-49360dcd7859","Type":"ContainerStarted","Data":"b7b005cbddca785ee079ec0c38d0de17634fb2c837d81c697602ddac52dbfb54"}
Sep 11 13:49:54.662771 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:54.662741    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-th29l" event={"ID":"61776a5c-0574-4877-af8e-f94a5309536f","Type":"ContainerStarted","Data":"c206a1359adcf88a2a593153c49394fc06f16292143a3073ba9a582bdff3149d"}
Sep 11 13:49:54.681380 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:54.681323    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pzd6j" podStartSLOduration=36.859016293 podStartE2EDuration="38.68130419s" totalImagesPullingTime="1.822287897s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:49:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:54.680281545 +0000 UTC m=+76.077624283" watchObservedRunningTime="2026-09-11 13:49:54.68130419 +0000 UTC m=+76.078646927"
Sep 11 13:49:55.667704 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:55.667671    2563 generic.go:356] "Generic (PLEG): container finished" podID="61776a5c-0574-4877-af8e-f94a5309536f" containerID="b1d1168b024f5d277254bfaa11ddec88dbac5a1037d0d2fe45363d10f33e42e2" exitCode=0
Sep 11 13:49:55.668093 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:55.667764    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-th29l" event={"ID":"61776a5c-0574-4877-af8e-f94a5309536f","Type":"ContainerDied","Data":"b1d1168b024f5d277254bfaa11ddec88dbac5a1037d0d2fe45363d10f33e42e2"}
Sep 11 13:49:56.677983 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:56.677928    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-th29l" event={"ID":"61776a5c-0574-4877-af8e-f94a5309536f","Type":"ContainerStarted","Data":"a61537695bd482b222bbaed84132b808319cd39767a1e82012191cbd69c36c9c"}
Sep 11 13:49:56.681414 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:56.678544    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-th29l" event={"ID":"61776a5c-0574-4877-af8e-f94a5309536f","Type":"ContainerStarted","Data":"4578adc396c07aaa53b31aee08cf16b5839102e53afc80bba5642d63c26d6f8a"}
Sep 11 13:49:56.704067 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:49:56.704015    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-th29l" podStartSLOduration=3.529593539 podStartE2EDuration="4.704000575s" totalImagesPullingTime="1.106248485s" totalInitContainerRuntime="68.158551ms" isStatefulPod=true podCreationTimestamp="2026-09-11 13:49:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:56.702791064 +0000 UTC m=+78.100133801" watchObservedRunningTime="2026-09-11 13:49:56.704000575 +0000 UTC m=+78.101343313"
Sep 11 13:50:04.589055 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:04.588929    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f77897478-vbwcm"
Sep 11 13:50:05.898878 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:05.898811    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6cfbdd748c-l979v" podUID="fac2fe28-c77e-4648-b789-b18824e80a3f" containerName="registry" containerID="cri-o://7cfec4c3758da380b8c10b55f35ce3edd89ce9f6846e05d864227bd51159aa0f" gracePeriod=30
Sep 11 13:50:06.134354 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.134331    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:50:06.235744 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.235673    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-certificates\") pod \"fac2fe28-c77e-4648-b789-b18824e80a3f\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") "
Sep 11 13:50:06.235744 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.235709    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls\") pod \"fac2fe28-c77e-4648-b789-b18824e80a3f\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") "
Sep 11 13:50:06.235744 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.235736    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fac2fe28-c77e-4648-b789-b18824e80a3f-image-registry-private-configuration\") pod \"fac2fe28-c77e-4648-b789-b18824e80a3f\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") "
Sep 11 13:50:06.236005 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.235767    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fac2fe28-c77e-4648-b789-b18824e80a3f-trusted-ca\") pod \"fac2fe28-c77e-4648-b789-b18824e80a3f\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") "
Sep 11 13:50:06.236005 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.235789    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4sxxp\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-kube-api-access-4sxxp\") pod \"fac2fe28-c77e-4648-b789-b18824e80a3f\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") "
Sep 11 13:50:06.236005 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.235898    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fac2fe28-c77e-4648-b789-b18824e80a3f-ca-trust-extracted\") pod \"fac2fe28-c77e-4648-b789-b18824e80a3f\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") "
Sep 11 13:50:06.236005 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.235951    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-bound-sa-token\") pod \"fac2fe28-c77e-4648-b789-b18824e80a3f\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") "
Sep 11 13:50:06.236005 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.235977    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fac2fe28-c77e-4648-b789-b18824e80a3f-installation-pull-secrets\") pod \"fac2fe28-c77e-4648-b789-b18824e80a3f\" (UID: \"fac2fe28-c77e-4648-b789-b18824e80a3f\") "
Sep 11 13:50:06.236245 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.236180    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fac2fe28-c77e-4648-b789-b18824e80a3f" (UID: "fac2fe28-c77e-4648-b789-b18824e80a3f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:06.236345 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.236327    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-certificates\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:50:06.236847 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.236783    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fac2fe28-c77e-4648-b789-b18824e80a3f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fac2fe28-c77e-4648-b789-b18824e80a3f" (UID: "fac2fe28-c77e-4648-b789-b18824e80a3f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:06.238404 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.238376    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fac2fe28-c77e-4648-b789-b18824e80a3f" (UID: "fac2fe28-c77e-4648-b789-b18824e80a3f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:06.238513 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.238402    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fac2fe28-c77e-4648-b789-b18824e80a3f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fac2fe28-c77e-4648-b789-b18824e80a3f" (UID: "fac2fe28-c77e-4648-b789-b18824e80a3f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:50:06.238733 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.238709    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-kube-api-access-4sxxp" (OuterVolumeSpecName: "kube-api-access-4sxxp") pod "fac2fe28-c77e-4648-b789-b18824e80a3f" (UID: "fac2fe28-c77e-4648-b789-b18824e80a3f"). InnerVolumeSpecName "kube-api-access-4sxxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:06.238820 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.238795    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fac2fe28-c77e-4648-b789-b18824e80a3f" (UID: "fac2fe28-c77e-4648-b789-b18824e80a3f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:06.238884 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.238824    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fac2fe28-c77e-4648-b789-b18824e80a3f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fac2fe28-c77e-4648-b789-b18824e80a3f" (UID: "fac2fe28-c77e-4648-b789-b18824e80a3f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:50:06.244739 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.244719    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fac2fe28-c77e-4648-b789-b18824e80a3f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fac2fe28-c77e-4648-b789-b18824e80a3f" (UID: "fac2fe28-c77e-4648-b789-b18824e80a3f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:06.336994 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.336921    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sxxp\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-kube-api-access-4sxxp\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:50:06.336994 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.336990    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fac2fe28-c77e-4648-b789-b18824e80a3f-ca-trust-extracted\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:50:06.336994 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.337000    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-bound-sa-token\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:50:06.337198 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.337009    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fac2fe28-c77e-4648-b789-b18824e80a3f-installation-pull-secrets\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:50:06.337198 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.337018    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fac2fe28-c77e-4648-b789-b18824e80a3f-registry-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:50:06.337198 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.337026    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fac2fe28-c77e-4648-b789-b18824e80a3f-image-registry-private-configuration\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:50:06.337198 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.337034    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fac2fe28-c77e-4648-b789-b18824e80a3f-trusted-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:50:06.714852 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.714819    2563 generic.go:356] "Generic (PLEG): container finished" podID="fac2fe28-c77e-4648-b789-b18824e80a3f" containerID="7cfec4c3758da380b8c10b55f35ce3edd89ce9f6846e05d864227bd51159aa0f" exitCode=0
Sep 11 13:50:06.714852 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.714856    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cfbdd748c-l979v" event={"ID":"fac2fe28-c77e-4648-b789-b18824e80a3f","Type":"ContainerDied","Data":"7cfec4c3758da380b8c10b55f35ce3edd89ce9f6846e05d864227bd51159aa0f"}
Sep 11 13:50:06.715049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.714866    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cfbdd748c-l979v"
Sep 11 13:50:06.715049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.714896    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cfbdd748c-l979v" event={"ID":"fac2fe28-c77e-4648-b789-b18824e80a3f","Type":"ContainerDied","Data":"3252793c23d35f114e9d57cc543bfa933a09367951bf1286f5662aae4902da35"}
Sep 11 13:50:06.715049 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.714918    2563 scope.go:117] "RemoveContainer" containerID="7cfec4c3758da380b8c10b55f35ce3edd89ce9f6846e05d864227bd51159aa0f"
Sep 11 13:50:06.726750 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.726729    2563 scope.go:117] "RemoveContainer" containerID="7cfec4c3758da380b8c10b55f35ce3edd89ce9f6846e05d864227bd51159aa0f"
Sep 11 13:50:06.727019 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:50:06.726998    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cfec4c3758da380b8c10b55f35ce3edd89ce9f6846e05d864227bd51159aa0f\": container with ID starting with 7cfec4c3758da380b8c10b55f35ce3edd89ce9f6846e05d864227bd51159aa0f not found: ID does not exist" containerID="7cfec4c3758da380b8c10b55f35ce3edd89ce9f6846e05d864227bd51159aa0f"
Sep 11 13:50:06.727098 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.727031    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cfec4c3758da380b8c10b55f35ce3edd89ce9f6846e05d864227bd51159aa0f"} err="failed to get container status \"7cfec4c3758da380b8c10b55f35ce3edd89ce9f6846e05d864227bd51159aa0f\": rpc error: code = NotFound desc = could not find container \"7cfec4c3758da380b8c10b55f35ce3edd89ce9f6846e05d864227bd51159aa0f\": container with ID starting with 7cfec4c3758da380b8c10b55f35ce3edd89ce9f6846e05d864227bd51159aa0f not found: ID does not exist"
Sep 11 13:50:06.737920 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.737896    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cfbdd748c-l979v"]
Sep 11 13:50:06.742322 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:06.742295    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6cfbdd748c-l979v"]
Sep 11 13:50:07.130116 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:07.130075    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fac2fe28-c77e-4648-b789-b18824e80a3f" path="/var/lib/kubelet/pods/fac2fe28-c77e-4648-b789-b18824e80a3f/volumes"
Sep 11 13:50:23.658007 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:50:23.657976    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lrb9b"
Sep 11 13:53:16.476116 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.476080    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"]
Sep 11 13:53:16.476711 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.476334    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fac2fe28-c77e-4648-b789-b18824e80a3f" containerName="registry"
Sep 11 13:53:16.476711 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.476345    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fac2fe28-c77e-4648-b789-b18824e80a3f" containerName="registry"
Sep 11 13:53:16.476711 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.476402    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fac2fe28-c77e-4648-b789-b18824e80a3f" containerName="registry"
Sep 11 13:53:16.478239 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.478224    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.480478 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.480460    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 13:53:16.482411 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.482396    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 13:53:16.483381 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.483322    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 13:53:16.483381 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.483359    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 13:53:16.483521 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.483496    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8wssf\""
Sep 11 13:53:16.504252 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.504215    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/513da08b-e7f7-4ad2-bffd-a0971bce095a-webhook-cert\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2x8r7\" (UID: \"513da08b-e7f7-4ad2-bffd-a0971bce095a\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.504252 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.504241    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/513da08b-e7f7-4ad2-bffd-a0971bce095a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2x8r7\" (UID: \"513da08b-e7f7-4ad2-bffd-a0971bce095a\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.504404 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.504268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fflz\" (UniqueName: \"kubernetes.io/projected/513da08b-e7f7-4ad2-bffd-a0971bce095a-kube-api-access-7fflz\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2x8r7\" (UID: \"513da08b-e7f7-4ad2-bffd-a0971bce095a\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.504404 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.504328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/513da08b-e7f7-4ad2-bffd-a0971bce095a-tmp\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2x8r7\" (UID: \"513da08b-e7f7-4ad2-bffd-a0971bce095a\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.504404 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.504368    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/513da08b-e7f7-4ad2-bffd-a0971bce095a-manifests\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2x8r7\" (UID: \"513da08b-e7f7-4ad2-bffd-a0971bce095a\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.507127 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.507088    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"]
Sep 11 13:53:16.605119 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.605096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/513da08b-e7f7-4ad2-bffd-a0971bce095a-tmp\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2x8r7\" (UID: \"513da08b-e7f7-4ad2-bffd-a0971bce095a\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.605247 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.605126    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/513da08b-e7f7-4ad2-bffd-a0971bce095a-manifests\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2x8r7\" (UID: \"513da08b-e7f7-4ad2-bffd-a0971bce095a\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.605247 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.605165    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/513da08b-e7f7-4ad2-bffd-a0971bce095a-webhook-cert\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2x8r7\" (UID: \"513da08b-e7f7-4ad2-bffd-a0971bce095a\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.605247 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.605199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/513da08b-e7f7-4ad2-bffd-a0971bce095a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2x8r7\" (UID: \"513da08b-e7f7-4ad2-bffd-a0971bce095a\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.605247 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.605221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fflz\" (UniqueName: \"kubernetes.io/projected/513da08b-e7f7-4ad2-bffd-a0971bce095a-kube-api-access-7fflz\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2x8r7\" (UID: \"513da08b-e7f7-4ad2-bffd-a0971bce095a\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.605556 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.605535    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/513da08b-e7f7-4ad2-bffd-a0971bce095a-manifests\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2x8r7\" (UID: \"513da08b-e7f7-4ad2-bffd-a0971bce095a\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.607559 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.607539    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/513da08b-e7f7-4ad2-bffd-a0971bce095a-tmp\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2x8r7\" (UID: \"513da08b-e7f7-4ad2-bffd-a0971bce095a\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.607611 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.607560    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/513da08b-e7f7-4ad2-bffd-a0971bce095a-webhook-cert\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2x8r7\" (UID: \"513da08b-e7f7-4ad2-bffd-a0971bce095a\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.607725 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.607710    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/513da08b-e7f7-4ad2-bffd-a0971bce095a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2x8r7\" (UID: \"513da08b-e7f7-4ad2-bffd-a0971bce095a\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.623480 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.623455    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fflz\" (UniqueName: \"kubernetes.io/projected/513da08b-e7f7-4ad2-bffd-a0971bce095a-kube-api-access-7fflz\") pod \"opendatahub-operator-controller-manager-85cf544d9c-2x8r7\" (UID: \"513da08b-e7f7-4ad2-bffd-a0971bce095a\") " pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:16.788405 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:16.788330    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:17.082411 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:53:17.082384    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod513da08b_e7f7_4ad2_bffd_a0971bce095a.slice/crio-66565c132142f018dc2b7815cbaa0df451745f19c3dc9baf9a5f1650f6820834 WatchSource:0}: Error finding container 66565c132142f018dc2b7815cbaa0df451745f19c3dc9baf9a5f1650f6820834: Status 404 returned error can't find the container with id 66565c132142f018dc2b7815cbaa0df451745f19c3dc9baf9a5f1650f6820834
Sep 11 13:53:17.085862 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:17.085840    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"]
Sep 11 13:53:17.341982 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:17.341900    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7" event={"ID":"513da08b-e7f7-4ad2-bffd-a0971bce095a","Type":"ContainerStarted","Data":"66565c132142f018dc2b7815cbaa0df451745f19c3dc9baf9a5f1650f6820834"}
Sep 11 13:53:20.355325 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:20.355288    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7" event={"ID":"513da08b-e7f7-4ad2-bffd-a0971bce095a","Type":"ContainerStarted","Data":"3bc5658f35b8ad608ae932e7fed203434cc3899cae6f60aea81571746f7decd7"}
Sep 11 13:53:21.360360 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:21.360334    2563 generic.go:356] "Generic (PLEG): container finished" podID="513da08b-e7f7-4ad2-bffd-a0971bce095a" containerID="3bc5658f35b8ad608ae932e7fed203434cc3899cae6f60aea81571746f7decd7" exitCode=0
Sep 11 13:53:21.360698 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:21.360386    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7" event={"ID":"513da08b-e7f7-4ad2-bffd-a0971bce095a","Type":"ContainerDied","Data":"3bc5658f35b8ad608ae932e7fed203434cc3899cae6f60aea81571746f7decd7"}
Sep 11 13:53:22.365947 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:22.365908    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7" event={"ID":"513da08b-e7f7-4ad2-bffd-a0971bce095a","Type":"ContainerStarted","Data":"480a54a04e7f4f478593b319295615551cb1920a0617544571eb490235df7c3f"}
Sep 11 13:53:22.366302 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:22.366047    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:22.392032 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:22.391981    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7" podStartSLOduration=1.6451366250000001 podStartE2EDuration="6.391961119s" totalImagesPullingTime="3.668717763s" totalInitContainerRuntime="1.078106731s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:22.390433153 +0000 UTC m=+283.787775890" watchObservedRunningTime="2026-09-11 13:53:22.391961119 +0000 UTC m=+283.789303876"
Sep 11 13:53:33.371542 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:33.371507    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-85cf544d9c-2x8r7"
Sep 11 13:53:36.643699 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.643669    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr"]
Sep 11 13:53:36.650384 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.650362    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr"
Sep 11 13:53:36.654697 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.654676    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 13:53:36.654914 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.654897    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 13:53:36.655362 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.655344    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 13:53:36.655510 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.655435    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 13:53:36.658171 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.658149    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr"]
Sep 11 13:53:36.742867 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.742834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s86zq\" (UniqueName: \"kubernetes.io/projected/1e0ee83b-71a7-47ae-b510-631402cc6799-kube-api-access-s86zq\") pod \"kube-auth-proxy-846c6f95fb-zqnpr\" (UID: \"1e0ee83b-71a7-47ae-b510-631402cc6799\") " pod="openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr"
Sep 11 13:53:36.743015 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.742882    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e0ee83b-71a7-47ae-b510-631402cc6799-tmp\") pod \"kube-auth-proxy-846c6f95fb-zqnpr\" (UID: \"1e0ee83b-71a7-47ae-b510-631402cc6799\") " pod="openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr"
Sep 11 13:53:36.743015 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.742939    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e0ee83b-71a7-47ae-b510-631402cc6799-tls-certs\") pod \"kube-auth-proxy-846c6f95fb-zqnpr\" (UID: \"1e0ee83b-71a7-47ae-b510-631402cc6799\") " pod="openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr"
Sep 11 13:53:36.843832 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.843791    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e0ee83b-71a7-47ae-b510-631402cc6799-tmp\") pod \"kube-auth-proxy-846c6f95fb-zqnpr\" (UID: \"1e0ee83b-71a7-47ae-b510-631402cc6799\") " pod="openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr"
Sep 11 13:53:36.843832 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.843837    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e0ee83b-71a7-47ae-b510-631402cc6799-tls-certs\") pod \"kube-auth-proxy-846c6f95fb-zqnpr\" (UID: \"1e0ee83b-71a7-47ae-b510-631402cc6799\") " pod="openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr"
Sep 11 13:53:36.844026 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.843891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s86zq\" (UniqueName: \"kubernetes.io/projected/1e0ee83b-71a7-47ae-b510-631402cc6799-kube-api-access-s86zq\") pod \"kube-auth-proxy-846c6f95fb-zqnpr\" (UID: \"1e0ee83b-71a7-47ae-b510-631402cc6799\") " pod="openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr"
Sep 11 13:53:36.846254 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.846231    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e0ee83b-71a7-47ae-b510-631402cc6799-tmp\") pod \"kube-auth-proxy-846c6f95fb-zqnpr\" (UID: \"1e0ee83b-71a7-47ae-b510-631402cc6799\") " pod="openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr"
Sep 11 13:53:36.846549 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.846529    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e0ee83b-71a7-47ae-b510-631402cc6799-tls-certs\") pod \"kube-auth-proxy-846c6f95fb-zqnpr\" (UID: \"1e0ee83b-71a7-47ae-b510-631402cc6799\") " pod="openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr"
Sep 11 13:53:36.868904 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.868880    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s86zq\" (UniqueName: \"kubernetes.io/projected/1e0ee83b-71a7-47ae-b510-631402cc6799-kube-api-access-s86zq\") pod \"kube-auth-proxy-846c6f95fb-zqnpr\" (UID: \"1e0ee83b-71a7-47ae-b510-631402cc6799\") " pod="openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr"
Sep 11 13:53:36.960615 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:36.960545    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr"
Sep 11 13:53:37.234353 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:37.234310    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr"]
Sep 11 13:53:37.236900 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:53:37.236872    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e0ee83b_71a7_47ae_b510_631402cc6799.slice/crio-af58fe4d45e8b9a480eb699cccd1049b32721ccd9da2962d7407044cc40a1e60 WatchSource:0}: Error finding container af58fe4d45e8b9a480eb699cccd1049b32721ccd9da2962d7407044cc40a1e60: Status 404 returned error can't find the container with id af58fe4d45e8b9a480eb699cccd1049b32721ccd9da2962d7407044cc40a1e60
Sep 11 13:53:37.417995 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:37.417962    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr" event={"ID":"1e0ee83b-71a7-47ae-b510-631402cc6799","Type":"ContainerStarted","Data":"af58fe4d45e8b9a480eb699cccd1049b32721ccd9da2962d7407044cc40a1e60"}
Sep 11 13:53:37.992325 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:37.992280    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"]
Sep 11 13:53:37.994653 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:37.994552    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:37.996981 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:37.996844    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 11 13:53:37.996981 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:37.996857    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 13:53:37.997229 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:37.997211    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 11 13:53:37.997499 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:37.997396    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 11 13:53:37.997499 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:37.997427    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zzskc\""
Sep 11 13:53:38.009725 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.009670    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"]
Sep 11 13:53:38.055260 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.055174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.055260 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.055230    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.055463 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.055272    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.055463 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.055300    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs7dp\" (UniqueName: \"kubernetes.io/projected/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-kube-api-access-qs7dp\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.055463 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.055352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.055463 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.055374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.055689 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.055479    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.156944 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.156670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.156944 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.156718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.156944 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.156782    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.156944 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.156809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.156944 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.156848    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.156944 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.156889    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.156944 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.156915    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs7dp\" (UniqueName: \"kubernetes.io/projected/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-kube-api-access-qs7dp\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.157913 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.157856    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.159791 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.159766    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.160248 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.160192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.160392 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.160371    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.160774 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.160751    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.166090 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.166067    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.166518 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.166497    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs7dp\" (UniqueName: \"kubernetes.io/projected/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-kube-api-access-qs7dp\") pod \"istiod-openshift-gateway-7b7995d5cd-8c8qv\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.307281 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.307246    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:38.592956 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:38.592843    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"]
Sep 11 13:53:39.836344 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:53:39.836304    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d0aa077_2b5a_4a22_a19b_ce499f3afa4d.slice/crio-8907fdc1a15d432e80518c8d1c1179cacbd7a0681eed9b56fbf886fe7179fea6 WatchSource:0}: Error finding container 8907fdc1a15d432e80518c8d1c1179cacbd7a0681eed9b56fbf886fe7179fea6: Status 404 returned error can't find the container with id 8907fdc1a15d432e80518c8d1c1179cacbd7a0681eed9b56fbf886fe7179fea6
Sep 11 13:53:39.857720 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:39.857699    2563 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 13:53:39.891329 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:39.891304    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 13:53:40.431576 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:40.431542    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv" event={"ID":"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d","Type":"ContainerStarted","Data":"8907fdc1a15d432e80518c8d1c1179cacbd7a0681eed9b56fbf886fe7179fea6"}
Sep 11 13:53:40.432981 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:40.432955    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr" event={"ID":"1e0ee83b-71a7-47ae-b510-631402cc6799","Type":"ContainerStarted","Data":"9937b65c70c230a097e7c2e0c5d18183198e838699233a28fad1761b981106c5"}
Sep 11 13:53:45.004475 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:45.004439    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:53:45.004732 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:45.004507    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:53:45.453814 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:45.453781    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv" event={"ID":"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d","Type":"ContainerStarted","Data":"a2dc4deb380b638abdd7372ca7fa315433704a848c28602ce386691906b6c0d5"}
Sep 11 13:53:45.453984 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:45.453848    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:45.472068 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:45.472015    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-846c6f95fb-zqnpr" podStartSLOduration=6.821708375 podStartE2EDuration="9.472000518s" totalImagesPullingTime="2.650292143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:40.449005214 +0000 UTC m=+301.846347952" watchObservedRunningTime="2026-09-11 13:53:45.472000518 +0000 UTC m=+306.869343255"
Sep 11 13:53:45.472205 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:45.472124    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv" podStartSLOduration=3.306706108 podStartE2EDuration="8.472117985s" totalImagesPullingTime="5.165411877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:45.471252219 +0000 UTC m=+306.868594956" watchObservedRunningTime="2026-09-11 13:53:45.472117985 +0000 UTC m=+306.869460736"
Sep 11 13:53:46.459127 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:46.459097    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:53.930016 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:53.929982    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"]
Sep 11 13:53:53.934316 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:53.934298    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:53.948052 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:53.948021    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"]
Sep 11 13:53:53.982392 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:53.982362    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02021ad7-a325-4bda-8928-642506308f2c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:53.982530 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:53.982412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvqrm\" (UniqueName: \"kubernetes.io/projected/02021ad7-a325-4bda-8928-642506308f2c-kube-api-access-pvqrm\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:53.982530 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:53.982464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02021ad7-a325-4bda-8928-642506308f2c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:53.982530 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:53.982500    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02021ad7-a325-4bda-8928-642506308f2c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:53.982678 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:53.982559    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02021ad7-a325-4bda-8928-642506308f2c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:53.982678 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:53.982586    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02021ad7-a325-4bda-8928-642506308f2c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:53.982678 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:53.982604    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02021ad7-a325-4bda-8928-642506308f2c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.083646 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.083595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02021ad7-a325-4bda-8928-642506308f2c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.083646 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.083650    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02021ad7-a325-4bda-8928-642506308f2c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.083898 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.083681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02021ad7-a325-4bda-8928-642506308f2c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.083898 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.083713    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02021ad7-a325-4bda-8928-642506308f2c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.083898 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.083730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02021ad7-a325-4bda-8928-642506308f2c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.083898 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.083751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02021ad7-a325-4bda-8928-642506308f2c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.083898 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.083809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvqrm\" (UniqueName: \"kubernetes.io/projected/02021ad7-a325-4bda-8928-642506308f2c-kube-api-access-pvqrm\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.084609 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.084538    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02021ad7-a325-4bda-8928-642506308f2c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.086264 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.086238    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02021ad7-a325-4bda-8928-642506308f2c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.086772 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.086748    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02021ad7-a325-4bda-8928-642506308f2c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.086865 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.086822    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02021ad7-a325-4bda-8928-642506308f2c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.086910 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.086870    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02021ad7-a325-4bda-8928-642506308f2c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.091722 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.091692    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvqrm\" (UniqueName: \"kubernetes.io/projected/02021ad7-a325-4bda-8928-642506308f2c-kube-api-access-pvqrm\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.092342 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.092316    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02021ad7-a325-4bda-8928-642506308f2c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-bvw27\" (UID: \"02021ad7-a325-4bda-8928-642506308f2c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.243148 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.243063    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:54.520863 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.520713    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"]
Sep 11 13:53:54.523237 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:53:54.523205    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02021ad7_a325_4bda_8928_642506308f2c.slice/crio-36d2025cbe855b07429dbba5bedda8ad8881f554236117fd01a477f831e40280 WatchSource:0}: Error finding container 36d2025cbe855b07429dbba5bedda8ad8881f554236117fd01a477f831e40280: Status 404 returned error can't find the container with id 36d2025cbe855b07429dbba5bedda8ad8881f554236117fd01a477f831e40280
Sep 11 13:53:54.526288 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.526257    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:53:54.526370 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:54.526329    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:53:55.490237 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.490204    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27" event={"ID":"02021ad7-a325-4bda-8928-642506308f2c","Type":"ContainerStarted","Data":"6c04aadf85ab2a73b8153073a07a94ced60c21f5e02ddd5625594027ff6db0bb"}
Sep 11 13:53:55.490725 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.490257    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27" event={"ID":"02021ad7-a325-4bda-8928-642506308f2c","Type":"ContainerStarted","Data":"36d2025cbe855b07429dbba5bedda8ad8881f554236117fd01a477f831e40280"}
Sep 11 13:53:55.490725 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.490362    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:55.492103 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.492084    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27"
Sep 11 13:53:55.512204 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.512157    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bvw27" podStartSLOduration=2.512142963 podStartE2EDuration="2.512142963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:55.510418132 +0000 UTC m=+316.907760869" watchObservedRunningTime="2026-09-11 13:53:55.512142963 +0000 UTC m=+316.909485700"
Sep 11 13:53:55.570860 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.570817    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"]
Sep 11 13:53:55.571056 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.571034    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv" podUID="0d0aa077-2b5a-4a22-a19b-ce499f3afa4d" containerName="discovery" containerID="cri-o://a2dc4deb380b638abdd7372ca7fa315433704a848c28602ce386691906b6c0d5" gracePeriod=30
Sep 11 13:53:55.830842 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.830821    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:55.901547 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.901520    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-csr-dns-cert\") pod \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") "
Sep 11 13:53:55.901547 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.901552    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-cacerts\") pod \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") "
Sep 11 13:53:55.901788 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.901574    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-csr-ca-configmap\") pod \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") "
Sep 11 13:53:55.901788 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.901603    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs7dp\" (UniqueName: \"kubernetes.io/projected/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-kube-api-access-qs7dp\") pod \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") "
Sep 11 13:53:55.902027 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.901996    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "0d0aa077-2b5a-4a22-a19b-ce499f3afa4d" (UID: "0d0aa077-2b5a-4a22-a19b-ce499f3afa4d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:53:55.902120 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.902057    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-local-certs\") pod \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") "
Sep 11 13:53:55.902175 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.902147    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-token\") pod \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") "
Sep 11 13:53:55.902227 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.902186    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-kubeconfig\") pod \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\" (UID: \"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d\") "
Sep 11 13:53:55.902419 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.902398    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-csr-ca-configmap\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:53:55.904309 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.904267    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-cacerts" (OuterVolumeSpecName: "cacerts") pod "0d0aa077-2b5a-4a22-a19b-ce499f3afa4d" (UID: "0d0aa077-2b5a-4a22-a19b-ce499f3afa4d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:53:55.904555 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.904523    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-local-certs" (OuterVolumeSpecName: "local-certs") pod "0d0aa077-2b5a-4a22-a19b-ce499f3afa4d" (UID: "0d0aa077-2b5a-4a22-a19b-ce499f3afa4d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:53:55.904655 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.904563    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "0d0aa077-2b5a-4a22-a19b-ce499f3afa4d" (UID: "0d0aa077-2b5a-4a22-a19b-ce499f3afa4d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:53:55.904655 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.904583    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-kube-api-access-qs7dp" (OuterVolumeSpecName: "kube-api-access-qs7dp") pod "0d0aa077-2b5a-4a22-a19b-ce499f3afa4d" (UID: "0d0aa077-2b5a-4a22-a19b-ce499f3afa4d"). InnerVolumeSpecName "kube-api-access-qs7dp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:53:55.904779 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.904702    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-token" (OuterVolumeSpecName: "istio-token") pod "0d0aa077-2b5a-4a22-a19b-ce499f3afa4d" (UID: "0d0aa077-2b5a-4a22-a19b-ce499f3afa4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:53:55.904833 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:55.904816    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "0d0aa077-2b5a-4a22-a19b-ce499f3afa4d" (UID: "0d0aa077-2b5a-4a22-a19b-ce499f3afa4d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:53:56.003337 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.003312    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-csr-dns-cert\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:53:56.003337 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.003332    2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-cacerts\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:53:56.003337 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.003342    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qs7dp\" (UniqueName: \"kubernetes.io/projected/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-kube-api-access-qs7dp\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:53:56.003512 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.003351    2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-local-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:53:56.003512 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.003360    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-token\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:53:56.003512 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.003368    2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d-istio-kubeconfig\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:53:56.495019 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.494981    2563 generic.go:356] "Generic (PLEG): container finished" podID="0d0aa077-2b5a-4a22-a19b-ce499f3afa4d" containerID="a2dc4deb380b638abdd7372ca7fa315433704a848c28602ce386691906b6c0d5" exitCode=0
Sep 11 13:53:56.495451 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.495025    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"
Sep 11 13:53:56.495451 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.495059    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv" event={"ID":"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d","Type":"ContainerDied","Data":"a2dc4deb380b638abdd7372ca7fa315433704a848c28602ce386691906b6c0d5"}
Sep 11 13:53:56.495451 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.495108    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv" event={"ID":"0d0aa077-2b5a-4a22-a19b-ce499f3afa4d","Type":"ContainerDied","Data":"8907fdc1a15d432e80518c8d1c1179cacbd7a0681eed9b56fbf886fe7179fea6"}
Sep 11 13:53:56.495451 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.495127    2563 scope.go:117] "RemoveContainer" containerID="a2dc4deb380b638abdd7372ca7fa315433704a848c28602ce386691906b6c0d5"
Sep 11 13:53:56.505894 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.505879    2563 scope.go:117] "RemoveContainer" containerID="a2dc4deb380b638abdd7372ca7fa315433704a848c28602ce386691906b6c0d5"
Sep 11 13:53:56.506186 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:53:56.506164    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2dc4deb380b638abdd7372ca7fa315433704a848c28602ce386691906b6c0d5\": container with ID starting with a2dc4deb380b638abdd7372ca7fa315433704a848c28602ce386691906b6c0d5 not found: ID does not exist" containerID="a2dc4deb380b638abdd7372ca7fa315433704a848c28602ce386691906b6c0d5"
Sep 11 13:53:56.506257 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.506194    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2dc4deb380b638abdd7372ca7fa315433704a848c28602ce386691906b6c0d5"} err="failed to get container status \"a2dc4deb380b638abdd7372ca7fa315433704a848c28602ce386691906b6c0d5\": rpc error: code = NotFound desc = could not find container \"a2dc4deb380b638abdd7372ca7fa315433704a848c28602ce386691906b6c0d5\": container with ID starting with a2dc4deb380b638abdd7372ca7fa315433704a848c28602ce386691906b6c0d5 not found: ID does not exist"
Sep 11 13:53:56.559944 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.559916    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"]
Sep 11 13:53:56.572061 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.572039    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8c8qv"]
Sep 11 13:53:56.809508 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.809466    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf"]
Sep 11 13:53:56.809832 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.809815    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d0aa077-2b5a-4a22-a19b-ce499f3afa4d" containerName="discovery"
Sep 11 13:53:56.809927 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.809834    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d0aa077-2b5a-4a22-a19b-ce499f3afa4d" containerName="discovery"
Sep 11 13:53:56.809927 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.809920    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d0aa077-2b5a-4a22-a19b-ce499f3afa4d" containerName="discovery"
Sep 11 13:53:56.814349 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.814327    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf"
Sep 11 13:53:56.819810 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.819789    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-wlk26\""
Sep 11 13:53:56.819928 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.819886    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 11 13:53:56.832575 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.832553    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf"]
Sep 11 13:53:56.909851 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.909816    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e965efb-8de7-493e-9d18-4eb41c1250b2-cert\") pod \"kserve-controller-manager-d6f4b65b8-6ljtf\" (UID: \"8e965efb-8de7-493e-9d18-4eb41c1250b2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf"
Sep 11 13:53:56.910012 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:56.909859    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49drj\" (UniqueName: \"kubernetes.io/projected/8e965efb-8de7-493e-9d18-4eb41c1250b2-kube-api-access-49drj\") pod \"kserve-controller-manager-d6f4b65b8-6ljtf\" (UID: \"8e965efb-8de7-493e-9d18-4eb41c1250b2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf"
Sep 11 13:53:57.011021 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:57.010980    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e965efb-8de7-493e-9d18-4eb41c1250b2-cert\") pod \"kserve-controller-manager-d6f4b65b8-6ljtf\" (UID: \"8e965efb-8de7-493e-9d18-4eb41c1250b2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf"
Sep 11 13:53:57.011202 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:57.011042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49drj\" (UniqueName: \"kubernetes.io/projected/8e965efb-8de7-493e-9d18-4eb41c1250b2-kube-api-access-49drj\") pod \"kserve-controller-manager-d6f4b65b8-6ljtf\" (UID: \"8e965efb-8de7-493e-9d18-4eb41c1250b2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf"
Sep 11 13:53:57.013586 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:57.013561    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e965efb-8de7-493e-9d18-4eb41c1250b2-cert\") pod \"kserve-controller-manager-d6f4b65b8-6ljtf\" (UID: \"8e965efb-8de7-493e-9d18-4eb41c1250b2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf"
Sep 11 13:53:57.023690 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:57.023667    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49drj\" (UniqueName: \"kubernetes.io/projected/8e965efb-8de7-493e-9d18-4eb41c1250b2-kube-api-access-49drj\") pod \"kserve-controller-manager-d6f4b65b8-6ljtf\" (UID: \"8e965efb-8de7-493e-9d18-4eb41c1250b2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf"
Sep 11 13:53:57.123837 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:57.123760    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf"
Sep 11 13:53:57.130304 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:57.130280    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d0aa077-2b5a-4a22-a19b-ce499f3afa4d" path="/var/lib/kubelet/pods/0d0aa077-2b5a-4a22-a19b-ce499f3afa4d/volumes"
Sep 11 13:53:57.427795 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:57.427704    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf"]
Sep 11 13:53:57.431198 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:53:57.431173    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e965efb_8de7_493e_9d18_4eb41c1250b2.slice/crio-5fbb3b1577566567dd1f34d3804d45143fb2aeb99266e6af60d6ac8df37b2b93 WatchSource:0}: Error finding container 5fbb3b1577566567dd1f34d3804d45143fb2aeb99266e6af60d6ac8df37b2b93: Status 404 returned error can't find the container with id 5fbb3b1577566567dd1f34d3804d45143fb2aeb99266e6af60d6ac8df37b2b93
Sep 11 13:53:57.432566 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:57.432550    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:53:57.500982 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:53:57.500940    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf" event={"ID":"8e965efb-8de7-493e-9d18-4eb41c1250b2","Type":"ContainerStarted","Data":"5fbb3b1577566567dd1f34d3804d45143fb2aeb99266e6af60d6ac8df37b2b93"}
Sep 11 13:54:00.515001 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:54:00.514969    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf" event={"ID":"8e965efb-8de7-493e-9d18-4eb41c1250b2","Type":"ContainerStarted","Data":"fe4ee2d7642cd91042c30c2c105833ce1303741bf1434210bc5ee5bbb39e4b6e"}
Sep 11 13:54:00.515396 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:54:00.515072    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf"
Sep 11 13:54:00.530190 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:54:00.530144    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf" podStartSLOduration=1.9776815939999999 podStartE2EDuration="4.530129081s" totalImagesPullingTime="2.552447487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:00.530092851 +0000 UTC m=+321.927435613" watchObservedRunningTime="2026-09-11 13:54:00.530129081 +0000 UTC m=+321.927471819"
Sep 11 13:54:31.524872 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:54:31.524793    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6ljtf"
Sep 11 13:55:06.796894 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:06.796859    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"]
Sep 11 13:55:06.799963 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:06.799947    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:06.802126 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:06.802084    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 13:55:06.802126 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:06.802116    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-882pd\""
Sep 11 13:55:06.802309 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:06.802214    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 13:55:06.816207 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:06.816171    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"]
Sep 11 13:55:06.911243 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:06.911211    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/454324cf-6a82-48b2-8bf8-8adf8703ee30-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" (UID: \"454324cf-6a82-48b2-8bf8-8adf8703ee30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:06.911367 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:06.911249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvb5c\" (UniqueName: \"kubernetes.io/projected/454324cf-6a82-48b2-8bf8-8adf8703ee30-kube-api-access-dvb5c\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" (UID: \"454324cf-6a82-48b2-8bf8-8adf8703ee30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:07.012153 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:07.012117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/454324cf-6a82-48b2-8bf8-8adf8703ee30-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" (UID: \"454324cf-6a82-48b2-8bf8-8adf8703ee30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:07.012310 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:07.012171    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvb5c\" (UniqueName: \"kubernetes.io/projected/454324cf-6a82-48b2-8bf8-8adf8703ee30-kube-api-access-dvb5c\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" (UID: \"454324cf-6a82-48b2-8bf8-8adf8703ee30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:07.012487 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:07.012468    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/454324cf-6a82-48b2-8bf8-8adf8703ee30-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" (UID: \"454324cf-6a82-48b2-8bf8-8adf8703ee30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:07.025068 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:07.025046    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvb5c\" (UniqueName: \"kubernetes.io/projected/454324cf-6a82-48b2-8bf8-8adf8703ee30-kube-api-access-dvb5c\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" (UID: \"454324cf-6a82-48b2-8bf8-8adf8703ee30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:07.109096 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:07.109012    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:07.389435 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:07.389411    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"]
Sep 11 13:55:07.392781 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:55:07.392744    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod454324cf_6a82_48b2_8bf8_8adf8703ee30.slice/crio-5a57dac29a9dcc4bb40a750845a066cc92213524d175f160e7ccd6a1ff09ad4f WatchSource:0}: Error finding container 5a57dac29a9dcc4bb40a750845a066cc92213524d175f160e7ccd6a1ff09ad4f: Status 404 returned error can't find the container with id 5a57dac29a9dcc4bb40a750845a066cc92213524d175f160e7ccd6a1ff09ad4f
Sep 11 13:55:07.768446 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:07.768369    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994" event={"ID":"454324cf-6a82-48b2-8bf8-8adf8703ee30","Type":"ContainerStarted","Data":"5a57dac29a9dcc4bb40a750845a066cc92213524d175f160e7ccd6a1ff09ad4f"}
Sep 11 13:55:11.674811 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:11.674783    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hxdgt"]
Sep 11 13:55:11.679709 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:11.679677    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hxdgt"
Sep 11 13:55:11.684337 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:11.684311    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-9xkk8\""
Sep 11 13:55:11.684464 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:11.684342    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 11 13:55:11.690407 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:11.690385    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hxdgt"]
Sep 11 13:55:11.750147 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:11.750116    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4882\" (UniqueName: \"kubernetes.io/projected/bd4dcebc-05a5-4526-a839-f76766b88025-kube-api-access-f4882\") pod \"dns-operator-controller-manager-7b84cd7c9c-hxdgt\" (UID: \"bd4dcebc-05a5-4526-a839-f76766b88025\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hxdgt"
Sep 11 13:55:11.851384 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:11.851356    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4882\" (UniqueName: \"kubernetes.io/projected/bd4dcebc-05a5-4526-a839-f76766b88025-kube-api-access-f4882\") pod \"dns-operator-controller-manager-7b84cd7c9c-hxdgt\" (UID: \"bd4dcebc-05a5-4526-a839-f76766b88025\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hxdgt"
Sep 11 13:55:11.861127 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:11.861106    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4882\" (UniqueName: \"kubernetes.io/projected/bd4dcebc-05a5-4526-a839-f76766b88025-kube-api-access-f4882\") pod \"dns-operator-controller-manager-7b84cd7c9c-hxdgt\" (UID: \"bd4dcebc-05a5-4526-a839-f76766b88025\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hxdgt"
Sep 11 13:55:11.990956 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:11.990872    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hxdgt"
Sep 11 13:55:12.262236 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:12.262215    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hxdgt"]
Sep 11 13:55:12.264281 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:55:12.264252    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd4dcebc_05a5_4526_a839_f76766b88025.slice/crio-91910e0747a57b4ca9755d3c7d744ed68dc7e9f7a13d039e4f3895b42b07723e WatchSource:0}: Error finding container 91910e0747a57b4ca9755d3c7d744ed68dc7e9f7a13d039e4f3895b42b07723e: Status 404 returned error can't find the container with id 91910e0747a57b4ca9755d3c7d744ed68dc7e9f7a13d039e4f3895b42b07723e
Sep 11 13:55:12.792988 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:12.792951    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hxdgt" event={"ID":"bd4dcebc-05a5-4526-a839-f76766b88025","Type":"ContainerStarted","Data":"91910e0747a57b4ca9755d3c7d744ed68dc7e9f7a13d039e4f3895b42b07723e"}
Sep 11 13:55:12.794511 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:12.794487    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994" event={"ID":"454324cf-6a82-48b2-8bf8-8adf8703ee30","Type":"ContainerStarted","Data":"1b5d10643cb687e7a8cda79318ea8077ac614f856643816c148826fbd8380da5"}
Sep 11 13:55:12.794675 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:12.794610    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:12.817246 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:12.817191    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994" podStartSLOduration=2.440058914 podStartE2EDuration="6.817176697s" totalImagesPullingTime="4.377117783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:55:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:55:12.815550001 +0000 UTC m=+394.212892750" watchObservedRunningTime="2026-09-11 13:55:12.817176697 +0000 UTC m=+394.214519446"
Sep 11 13:55:13.671438 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:13.671390    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"]
Sep 11 13:55:13.675672 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:13.675644    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:13.679075 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:13.679052    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-rzn8n\""
Sep 11 13:55:13.688943 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:13.688915    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"]
Sep 11 13:55:13.767312 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:13.767287    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjhk4\" (UniqueName: \"kubernetes.io/projected/3d93baa1-fc65-48bc-9e5f-f006197f8624-kube-api-access-bjhk4\") pod \"limitador-operator-controller-manager-74495c69f8-82tzm\" (UID: \"3d93baa1-fc65-48bc-9e5f-f006197f8624\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:13.800447 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:13.800418    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hxdgt" event={"ID":"bd4dcebc-05a5-4526-a839-f76766b88025","Type":"ContainerStarted","Data":"91e10f2817f659b088be1edfaa2b3d72a70d83607cd862f3a2f1a7d53fd639fa"}
Sep 11 13:55:13.800861 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:13.800508    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hxdgt"
Sep 11 13:55:13.827078 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:13.827033    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hxdgt" podStartSLOduration=1.404917113 podStartE2EDuration="2.827015674s" totalImagesPullingTime="1.422098561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:55:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:55:13.826032756 +0000 UTC m=+395.223375494" watchObservedRunningTime="2026-09-11 13:55:13.827015674 +0000 UTC m=+395.224358410"
Sep 11 13:55:13.868424 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:13.868399    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjhk4\" (UniqueName: \"kubernetes.io/projected/3d93baa1-fc65-48bc-9e5f-f006197f8624-kube-api-access-bjhk4\") pod \"limitador-operator-controller-manager-74495c69f8-82tzm\" (UID: \"3d93baa1-fc65-48bc-9e5f-f006197f8624\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:13.883454 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:13.883430    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjhk4\" (UniqueName: \"kubernetes.io/projected/3d93baa1-fc65-48bc-9e5f-f006197f8624-kube-api-access-bjhk4\") pod \"limitador-operator-controller-manager-74495c69f8-82tzm\" (UID: \"3d93baa1-fc65-48bc-9e5f-f006197f8624\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:13.988965 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:13.988884    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:14.259530 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:14.259374    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"]
Sep 11 13:55:14.262122 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:55:14.262093    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d93baa1_fc65_48bc_9e5f_f006197f8624.slice/crio-9213aed6396ddd0e48809c0e58b7cf8b488bca5d5ab0070f769d7f42f1552cff WatchSource:0}: Error finding container 9213aed6396ddd0e48809c0e58b7cf8b488bca5d5ab0070f769d7f42f1552cff: Status 404 returned error can't find the container with id 9213aed6396ddd0e48809c0e58b7cf8b488bca5d5ab0070f769d7f42f1552cff
Sep 11 13:55:14.805973 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:14.805937    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm" event={"ID":"3d93baa1-fc65-48bc-9e5f-f006197f8624","Type":"ContainerStarted","Data":"9213aed6396ddd0e48809c0e58b7cf8b488bca5d5ab0070f769d7f42f1552cff"}
Sep 11 13:55:16.817317 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:16.817284    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm" event={"ID":"3d93baa1-fc65-48bc-9e5f-f006197f8624","Type":"ContainerStarted","Data":"779e6697f90d749686b889c800bead5b05ec24515a2a862d41e12ff83a9e20da"}
Sep 11 13:55:16.817788 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:16.817413    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:16.833865 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:16.833825    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm" podStartSLOduration=1.969950415 podStartE2EDuration="3.833808097s" totalImagesPullingTime="1.863857682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:55:16.832194613 +0000 UTC m=+398.229537350" watchObservedRunningTime="2026-09-11 13:55:16.833808097 +0000 UTC m=+398.231150834"
Sep 11 13:55:23.802650 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:23.802604    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:24.808200 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:24.808172    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hxdgt"
Sep 11 13:55:25.877172 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.877130    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"]
Sep 11 13:55:25.877620 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.877417    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994" podUID="454324cf-6a82-48b2-8bf8-8adf8703ee30" containerName="manager" containerID="cri-o://1b5d10643cb687e7a8cda79318ea8077ac614f856643816c148826fbd8380da5" gracePeriod=2
Sep 11 13:55:25.890201 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.890175    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"]
Sep 11 13:55:25.901709 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:25.901677    2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"454324cf-6a82-48b2-8bf8-8adf8703ee30\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-pb994\": pods \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:25.903510 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.903487    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"]
Sep 11 13:55:25.903740 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.903720    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm" podUID="3d93baa1-fc65-48bc-9e5f-f006197f8624" containerName="manager" containerID="cri-o://779e6697f90d749686b889c800bead5b05ec24515a2a862d41e12ff83a9e20da" gracePeriod=2
Sep 11 13:55:25.905698 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.905677    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:25.915681 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.915657    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"]
Sep 11 13:55:25.921780 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:25.921751    2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3d93baa1-fc65-48bc-9e5f-f006197f8624\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-82tzm\": pods \"limitador-operator-controller-manager-74495c69f8-82tzm\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:25.923620 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:25.923595    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="454324cf-6a82-48b2-8bf8-8adf8703ee30" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:25.925259 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:25.925238    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-82tzm\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="3d93baa1-fc65-48bc-9e5f-f006197f8624" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:25.926814 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:25.926787    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="454324cf-6a82-48b2-8bf8-8adf8703ee30" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:25.928262 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.928245    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-frz57"]
Sep 11 13:55:25.928498 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.928487    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d93baa1-fc65-48bc-9e5f-f006197f8624" containerName="manager"
Sep 11 13:55:25.928546 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.928499    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d93baa1-fc65-48bc-9e5f-f006197f8624" containerName="manager"
Sep 11 13:55:25.928546 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.928523    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="454324cf-6a82-48b2-8bf8-8adf8703ee30" containerName="manager"
Sep 11 13:55:25.928546 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.928529    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="454324cf-6a82-48b2-8bf8-8adf8703ee30" containerName="manager"
Sep 11 13:55:25.928749 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.928581    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="454324cf-6a82-48b2-8bf8-8adf8703ee30" containerName="manager"
Sep 11 13:55:25.928749 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.928590    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d93baa1-fc65-48bc-9e5f-f006197f8624" containerName="manager"
Sep 11 13:55:25.930093 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.930080    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-frz57"
Sep 11 13:55:25.931796 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:25.931768    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-82tzm\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="3d93baa1-fc65-48bc-9e5f-f006197f8624" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:25.933141 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:25.933123    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="454324cf-6a82-48b2-8bf8-8adf8703ee30" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:25.947275 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:25.947254    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-frz57"]
Sep 11 13:55:26.058806 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.058777    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jss6v\" (UniqueName: \"kubernetes.io/projected/7237fe7b-716c-4de1-93eb-5cdfdd681b88-kube-api-access-jss6v\") pod \"limitador-operator-controller-manager-74495c69f8-frz57\" (UID: \"7237fe7b-716c-4de1-93eb-5cdfdd681b88\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-frz57"
Sep 11 13:55:26.068459 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.068428    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"]
Sep 11 13:55:26.071246 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.071226    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"
Sep 11 13:55:26.073902 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:26.073858    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="454324cf-6a82-48b2-8bf8-8adf8703ee30" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:26.088703 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.088651    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"]
Sep 11 13:55:26.116700 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:26.116665    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-82tzm\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="3d93baa1-fc65-48bc-9e5f-f006197f8624" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:26.148667 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.148639    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:26.151209 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:26.151180    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-82tzm\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="3d93baa1-fc65-48bc-9e5f-f006197f8624" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:26.152532 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.152506    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:26.153165 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:26.153137    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="454324cf-6a82-48b2-8bf8-8adf8703ee30" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:26.155470 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:26.155443    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-82tzm\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="3d93baa1-fc65-48bc-9e5f-f006197f8624" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:26.157239 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:26.157208    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="454324cf-6a82-48b2-8bf8-8adf8703ee30" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:26.159180 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.159153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4cfst\" (UID: \"5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"
Sep 11 13:55:26.159268 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.159233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sc4h\" (UniqueName: \"kubernetes.io/projected/5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd-kube-api-access-8sc4h\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4cfst\" (UID: \"5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"
Sep 11 13:55:26.159315 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.159303    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jss6v\" (UniqueName: \"kubernetes.io/projected/7237fe7b-716c-4de1-93eb-5cdfdd681b88-kube-api-access-jss6v\") pod \"limitador-operator-controller-manager-74495c69f8-frz57\" (UID: \"7237fe7b-716c-4de1-93eb-5cdfdd681b88\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-frz57"
Sep 11 13:55:26.177522 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.177501    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jss6v\" (UniqueName: \"kubernetes.io/projected/7237fe7b-716c-4de1-93eb-5cdfdd681b88-kube-api-access-jss6v\") pod \"limitador-operator-controller-manager-74495c69f8-frz57\" (UID: \"7237fe7b-716c-4de1-93eb-5cdfdd681b88\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-frz57"
Sep 11 13:55:26.260094 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.260067    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvb5c\" (UniqueName: \"kubernetes.io/projected/454324cf-6a82-48b2-8bf8-8adf8703ee30-kube-api-access-dvb5c\") pod \"454324cf-6a82-48b2-8bf8-8adf8703ee30\" (UID: \"454324cf-6a82-48b2-8bf8-8adf8703ee30\") "
Sep 11 13:55:26.260266 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.260128    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjhk4\" (UniqueName: \"kubernetes.io/projected/3d93baa1-fc65-48bc-9e5f-f006197f8624-kube-api-access-bjhk4\") pod \"3d93baa1-fc65-48bc-9e5f-f006197f8624\" (UID: \"3d93baa1-fc65-48bc-9e5f-f006197f8624\") "
Sep 11 13:55:26.260266 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.260159    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/454324cf-6a82-48b2-8bf8-8adf8703ee30-extensions-socket-volume\") pod \"454324cf-6a82-48b2-8bf8-8adf8703ee30\" (UID: \"454324cf-6a82-48b2-8bf8-8adf8703ee30\") "
Sep 11 13:55:26.260266 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.260230    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4cfst\" (UID: \"5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"
Sep 11 13:55:26.260430 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.260363    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sc4h\" (UniqueName: \"kubernetes.io/projected/5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd-kube-api-access-8sc4h\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4cfst\" (UID: \"5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"
Sep 11 13:55:26.260614 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.260589    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4cfst\" (UID: \"5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"
Sep 11 13:55:26.260775 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.260748    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/454324cf-6a82-48b2-8bf8-8adf8703ee30-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "454324cf-6a82-48b2-8bf8-8adf8703ee30" (UID: "454324cf-6a82-48b2-8bf8-8adf8703ee30"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:55:26.262482 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.262461    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d93baa1-fc65-48bc-9e5f-f006197f8624-kube-api-access-bjhk4" (OuterVolumeSpecName: "kube-api-access-bjhk4") pod "3d93baa1-fc65-48bc-9e5f-f006197f8624" (UID: "3d93baa1-fc65-48bc-9e5f-f006197f8624"). InnerVolumeSpecName "kube-api-access-bjhk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:55:26.262569 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.262476    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/454324cf-6a82-48b2-8bf8-8adf8703ee30-kube-api-access-dvb5c" (OuterVolumeSpecName: "kube-api-access-dvb5c") pod "454324cf-6a82-48b2-8bf8-8adf8703ee30" (UID: "454324cf-6a82-48b2-8bf8-8adf8703ee30"). InnerVolumeSpecName "kube-api-access-dvb5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:55:26.270053 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.270029    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sc4h\" (UniqueName: \"kubernetes.io/projected/5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd-kube-api-access-8sc4h\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4cfst\" (UID: \"5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"
Sep 11 13:55:26.322822 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.322795    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-frz57"
Sep 11 13:55:26.361062 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.361031    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvb5c\" (UniqueName: \"kubernetes.io/projected/454324cf-6a82-48b2-8bf8-8adf8703ee30-kube-api-access-dvb5c\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:55:26.361062 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.361060    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjhk4\" (UniqueName: \"kubernetes.io/projected/3d93baa1-fc65-48bc-9e5f-f006197f8624-kube-api-access-bjhk4\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:55:26.361062 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.361069    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/454324cf-6a82-48b2-8bf8-8adf8703ee30-extensions-socket-volume\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 13:55:26.385029 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.384944    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"
Sep 11 13:55:26.608186 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.608152    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-frz57"]
Sep 11 13:55:26.611118 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:55:26.611098    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7237fe7b_716c_4de1_93eb_5cdfdd681b88.slice/crio-0880f3a8756a3326ff4d3f815379d4cd0291aac878fd42900c4805b5dc51c3a6 WatchSource:0}: Error finding container 0880f3a8756a3326ff4d3f815379d4cd0291aac878fd42900c4805b5dc51c3a6: Status 404 returned error can't find the container with id 0880f3a8756a3326ff4d3f815379d4cd0291aac878fd42900c4805b5dc51c3a6
Sep 11 13:55:26.679292 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.679266    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"]
Sep 11 13:55:26.683297 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:55:26.683276    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d1fb34a_8f9a_4ad2_8bf0_274a4d64c8fd.slice/crio-4fd84e5813a4d0f34d7700b3cf7bb4eeb6d8755a30e4d0fb2f1c28dd6c3839af WatchSource:0}: Error finding container 4fd84e5813a4d0f34d7700b3cf7bb4eeb6d8755a30e4d0fb2f1c28dd6c3839af: Status 404 returned error can't find the container with id 4fd84e5813a4d0f34d7700b3cf7bb4eeb6d8755a30e4d0fb2f1c28dd6c3839af
Sep 11 13:55:26.858155 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.858120    2563 generic.go:356] "Generic (PLEG): container finished" podID="454324cf-6a82-48b2-8bf8-8adf8703ee30" containerID="1b5d10643cb687e7a8cda79318ea8077ac614f856643816c148826fbd8380da5" exitCode=0
Sep 11 13:55:26.858311 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.858167    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:26.858311 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.858213    2563 scope.go:117] "RemoveContainer" containerID="1b5d10643cb687e7a8cda79318ea8077ac614f856643816c148826fbd8380da5"
Sep 11 13:55:26.859723 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.859691    2563 generic.go:356] "Generic (PLEG): container finished" podID="3d93baa1-fc65-48bc-9e5f-f006197f8624" containerID="779e6697f90d749686b889c800bead5b05ec24515a2a862d41e12ff83a9e20da" exitCode=0
Sep 11 13:55:26.859813 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.859766    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:26.860419 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:26.860393    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-82tzm\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="3d93baa1-fc65-48bc-9e5f-f006197f8624" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:26.861826 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.861789    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-frz57" event={"ID":"7237fe7b-716c-4de1-93eb-5cdfdd681b88","Type":"ContainerStarted","Data":"a16ef6f826887c23904bf80480bb1abbfffa06ab2fd69859364bba2d0d611e18"}
Sep 11 13:55:26.861944 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.861839    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-frz57" event={"ID":"7237fe7b-716c-4de1-93eb-5cdfdd681b88","Type":"ContainerStarted","Data":"0880f3a8756a3326ff4d3f815379d4cd0291aac878fd42900c4805b5dc51c3a6"}
Sep 11 13:55:26.861944 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.861872    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-frz57"
Sep 11 13:55:26.862301 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:26.862271    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="454324cf-6a82-48b2-8bf8-8adf8703ee30" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:26.863675 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.863650    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst" event={"ID":"5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd","Type":"ContainerStarted","Data":"dd233e34b97db389bb978948aa5566186e517d30fa321c2d0d20f07529e7387d"}
Sep 11 13:55:26.863778 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.863701    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst" event={"ID":"5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd","Type":"ContainerStarted","Data":"4fd84e5813a4d0f34d7700b3cf7bb4eeb6d8755a30e4d0fb2f1c28dd6c3839af"}
Sep 11 13:55:26.863778 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.863731    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"
Sep 11 13:55:26.867477 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.867457    2563 scope.go:117] "RemoveContainer" containerID="1b5d10643cb687e7a8cda79318ea8077ac614f856643816c148826fbd8380da5"
Sep 11 13:55:26.867850 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:26.867829    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b5d10643cb687e7a8cda79318ea8077ac614f856643816c148826fbd8380da5\": container with ID starting with 1b5d10643cb687e7a8cda79318ea8077ac614f856643816c148826fbd8380da5 not found: ID does not exist" containerID="1b5d10643cb687e7a8cda79318ea8077ac614f856643816c148826fbd8380da5"
Sep 11 13:55:26.867923 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.867857    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b5d10643cb687e7a8cda79318ea8077ac614f856643816c148826fbd8380da5"} err="failed to get container status \"1b5d10643cb687e7a8cda79318ea8077ac614f856643816c148826fbd8380da5\": rpc error: code = NotFound desc = could not find container \"1b5d10643cb687e7a8cda79318ea8077ac614f856643816c148826fbd8380da5\": container with ID starting with 1b5d10643cb687e7a8cda79318ea8077ac614f856643816c148826fbd8380da5 not found: ID does not exist"
Sep 11 13:55:26.867923 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.867880    2563 scope.go:117] "RemoveContainer" containerID="779e6697f90d749686b889c800bead5b05ec24515a2a862d41e12ff83a9e20da"
Sep 11 13:55:26.876006 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.875988    2563 scope.go:117] "RemoveContainer" containerID="779e6697f90d749686b889c800bead5b05ec24515a2a862d41e12ff83a9e20da"
Sep 11 13:55:26.876261 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:26.876241    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"779e6697f90d749686b889c800bead5b05ec24515a2a862d41e12ff83a9e20da\": container with ID starting with 779e6697f90d749686b889c800bead5b05ec24515a2a862d41e12ff83a9e20da not found: ID does not exist" containerID="779e6697f90d749686b889c800bead5b05ec24515a2a862d41e12ff83a9e20da"
Sep 11 13:55:26.876318 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.876270    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"779e6697f90d749686b889c800bead5b05ec24515a2a862d41e12ff83a9e20da"} err="failed to get container status \"779e6697f90d749686b889c800bead5b05ec24515a2a862d41e12ff83a9e20da\": rpc error: code = NotFound desc = could not find container \"779e6697f90d749686b889c800bead5b05ec24515a2a862d41e12ff83a9e20da\": container with ID starting with 779e6697f90d749686b889c800bead5b05ec24515a2a862d41e12ff83a9e20da not found: ID does not exist"
Sep 11 13:55:26.879850 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.879803    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-frz57" podStartSLOduration=1.879787753 podStartE2EDuration="1.879787753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:55:26.879576147 +0000 UTC m=+408.276918895" watchObservedRunningTime="2026-09-11 13:55:26.879787753 +0000 UTC m=+408.277130491"
Sep 11 13:55:26.881299 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:26.881276    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-82tzm\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="3d93baa1-fc65-48bc-9e5f-f006197f8624" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:26.882779 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:26.882759    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="454324cf-6a82-48b2-8bf8-8adf8703ee30" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:26.884138 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:26.884112    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-82tzm\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="3d93baa1-fc65-48bc-9e5f-f006197f8624" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-82tzm"
Sep 11 13:55:26.885516 ip-10-0-128-161 kubenswrapper[2563]: E0911 13:55:26.885494    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-pb994\" is forbidden: User \"system:node:ip-10-0-128-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-161.ec2.internal' and this object" podUID="454324cf-6a82-48b2-8bf8-8adf8703ee30" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pb994"
Sep 11 13:55:26.903738 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:26.903697    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst" podStartSLOduration=0.903684565 podStartE2EDuration="903.684565ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:55:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:55:26.90269262 +0000 UTC m=+408.300035357" watchObservedRunningTime="2026-09-11 13:55:26.903684565 +0000 UTC m=+408.301027303"
Sep 11 13:55:27.133296 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:27.133264    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d93baa1-fc65-48bc-9e5f-f006197f8624" path="/var/lib/kubelet/pods/3d93baa1-fc65-48bc-9e5f-f006197f8624/volumes"
Sep 11 13:55:27.133606 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:27.133594    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="454324cf-6a82-48b2-8bf8-8adf8703ee30" path="/var/lib/kubelet/pods/454324cf-6a82-48b2-8bf8-8adf8703ee30/volumes"
Sep 11 13:55:37.870770 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:37.870741    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"
Sep 11 13:55:37.871223 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:55:37.870807    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-frz57"
Sep 11 13:57:30.276229 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.276141    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-987c8478-jx8s2"]
Sep 11 13:57:30.279315 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.279293    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:30.282383 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.282352    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 13:57:30.282543 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.282428    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 13:57:30.282543 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.282470    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fjz8d\""
Sep 11 13:57:30.282543 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.282481    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 13:57:30.282727 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.282371    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:57:30.291226 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.291196    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-987c8478-jx8s2"]
Sep 11 13:57:30.347272 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.347237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/195f9393-d4e8-46a6-877f-ea37cbd766c2-metrics-certs\") pod \"maas-api-987c8478-jx8s2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") " pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:30.347272 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.347272    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kswb\" (UniqueName: \"kubernetes.io/projected/195f9393-d4e8-46a6-877f-ea37cbd766c2-kube-api-access-6kswb\") pod \"maas-api-987c8478-jx8s2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") " pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:30.347526 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.347309    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/195f9393-d4e8-46a6-877f-ea37cbd766c2-openshift-service-ca\") pod \"maas-api-987c8478-jx8s2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") " pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:30.347526 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.347392    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/195f9393-d4e8-46a6-877f-ea37cbd766c2-maas-api-tls\") pod \"maas-api-987c8478-jx8s2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") " pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:30.448029 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.447985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/195f9393-d4e8-46a6-877f-ea37cbd766c2-metrics-certs\") pod \"maas-api-987c8478-jx8s2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") " pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:30.448029 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.448028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kswb\" (UniqueName: \"kubernetes.io/projected/195f9393-d4e8-46a6-877f-ea37cbd766c2-kube-api-access-6kswb\") pod \"maas-api-987c8478-jx8s2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") " pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:30.448311 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.448063    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/195f9393-d4e8-46a6-877f-ea37cbd766c2-openshift-service-ca\") pod \"maas-api-987c8478-jx8s2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") " pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:30.448311 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.448097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/195f9393-d4e8-46a6-877f-ea37cbd766c2-maas-api-tls\") pod \"maas-api-987c8478-jx8s2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") " pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:30.448885 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.448863    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/195f9393-d4e8-46a6-877f-ea37cbd766c2-openshift-service-ca\") pod \"maas-api-987c8478-jx8s2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") " pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:30.451171 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.451141    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/195f9393-d4e8-46a6-877f-ea37cbd766c2-metrics-certs\") pod \"maas-api-987c8478-jx8s2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") " pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:30.451309 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.451141    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/195f9393-d4e8-46a6-877f-ea37cbd766c2-maas-api-tls\") pod \"maas-api-987c8478-jx8s2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") " pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:30.457020 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.456989    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kswb\" (UniqueName: \"kubernetes.io/projected/195f9393-d4e8-46a6-877f-ea37cbd766c2-kube-api-access-6kswb\") pod \"maas-api-987c8478-jx8s2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") " pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:30.590894 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.590850    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:30.869286 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:30.869254    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-987c8478-jx8s2"]
Sep 11 13:57:30.871503 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:57:30.871474    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod195f9393_d4e8_46a6_877f_ea37cbd766c2.slice/crio-4546ca4d1de1d3a0603feb9b2c97b3671edaf50c688208303e06174f6384745b WatchSource:0}: Error finding container 4546ca4d1de1d3a0603feb9b2c97b3671edaf50c688208303e06174f6384745b: Status 404 returned error can't find the container with id 4546ca4d1de1d3a0603feb9b2c97b3671edaf50c688208303e06174f6384745b
Sep 11 13:57:31.376710 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:31.376675    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2" event={"ID":"195f9393-d4e8-46a6-877f-ea37cbd766c2","Type":"ContainerStarted","Data":"4546ca4d1de1d3a0603feb9b2c97b3671edaf50c688208303e06174f6384745b"}
Sep 11 13:57:33.388487 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:33.388450    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2" event={"ID":"195f9393-d4e8-46a6-877f-ea37cbd766c2","Type":"ContainerStarted","Data":"15bc746b95f3c94c1ad275265bdb1d5021a4cf0d2f3191a4257b1953a4f67e5b"}
Sep 11 13:57:33.388921 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:33.388546    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:33.404456 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:33.404398    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2" podStartSLOduration=1.024186384 podStartE2EDuration="3.404381313s" totalImagesPullingTime="2.380194929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:33.403088585 +0000 UTC m=+534.800431314" watchObservedRunningTime="2026-09-11 13:57:33.404381313 +0000 UTC m=+534.801724050"
Sep 11 13:57:39.398566 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:39.398536    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 13:57:53.172428 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.172384    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"]
Sep 11 13:57:53.179783 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.179750    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.182246 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.182220    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 13:57:53.182449 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.182435    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 13:57:53.182574 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.182537    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 11 13:57:53.182657 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.182589    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-p2lbh\""
Sep 11 13:57:53.185368 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.185341    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"]
Sep 11 13:57:53.345606 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.345563    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94btv\" (UniqueName: \"kubernetes.io/projected/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-kube-api-access-94btv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.345830 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.345659    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.345830 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.345696    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.345830 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.345774    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.345830 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.345817    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.345983 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.345849    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.446291 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.446185    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94btv\" (UniqueName: \"kubernetes.io/projected/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-kube-api-access-94btv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.446291 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.446237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.446291 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.446260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.446291 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.446292    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.446737 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.446325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.446737 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.446362    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.446861 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.446840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.446917 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.446863    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.446917 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.446890    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.448863 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.448839    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.448980 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.448950    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.455042 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.455008    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94btv\" (UniqueName: \"kubernetes.io/projected/f2ede573-a192-4a2e-a5e7-7d57080ff0a4-kube-api-access-94btv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww\" (UID: \"f2ede573-a192-4a2e-a5e7-7d57080ff0a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.491761 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.491720    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:57:53.768564 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:53.768479    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"]
Sep 11 13:57:53.772806 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:57:53.772720    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2ede573_a192_4a2e_a5e7_7d57080ff0a4.slice/crio-8d607df5ad8d16bb7db408887378a6b69f772ebbdabf2a11e8424e584e16e106 WatchSource:0}: Error finding container 8d607df5ad8d16bb7db408887378a6b69f772ebbdabf2a11e8424e584e16e106: Status 404 returned error can't find the container with id 8d607df5ad8d16bb7db408887378a6b69f772ebbdabf2a11e8424e584e16e106
Sep 11 13:57:54.482888 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:54.482850    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww" event={"ID":"f2ede573-a192-4a2e-a5e7-7d57080ff0a4","Type":"ContainerStarted","Data":"8d607df5ad8d16bb7db408887378a6b69f772ebbdabf2a11e8424e584e16e106"}
Sep 11 13:57:55.489221 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:55.489166    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww" event={"ID":"f2ede573-a192-4a2e-a5e7-7d57080ff0a4","Type":"ContainerStarted","Data":"a7c8ba5736778d66714435660a6981456a42290a2d3f3f1da2d5f2c069e83f61"}
Sep 11 13:57:55.506141 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:57:55.506077    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww" podStartSLOduration=1.346409895 podStartE2EDuration="2.506059403s" totalImagesPullingTime="1.159649508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:57:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:55.504842394 +0000 UTC m=+556.902185134" watchObservedRunningTime="2026-09-11 13:57:55.506059403 +0000 UTC m=+556.903402140"
Sep 11 13:58:03.492834 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:03.492787    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:58:03.493256 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:03.492867    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:58:03.505826 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:03.505789    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:58:03.541157 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:03.541119    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww"
Sep 11 13:58:49.864248 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:49.864200    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"]
Sep 11 13:58:49.866551 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:49.866533    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:49.868824 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:49.868793    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 11 13:58:49.876098 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:49.876066    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"]
Sep 11 13:58:50.002912 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.002870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c155709a-b1b8-4f40-b0f4-96a43a176163-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.002912 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.002913    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c155709a-b1b8-4f40-b0f4-96a43a176163-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.003127 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.002935    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxcx2\" (UniqueName: \"kubernetes.io/projected/c155709a-b1b8-4f40-b0f4-96a43a176163-kube-api-access-zxcx2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.003127 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.002969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c155709a-b1b8-4f40-b0f4-96a43a176163-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.003127 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.003015    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c155709a-b1b8-4f40-b0f4-96a43a176163-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.003127 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.003034    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c155709a-b1b8-4f40-b0f4-96a43a176163-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.103919 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.103874    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c155709a-b1b8-4f40-b0f4-96a43a176163-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.103919 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.103918    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c155709a-b1b8-4f40-b0f4-96a43a176163-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.104138 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.103944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxcx2\" (UniqueName: \"kubernetes.io/projected/c155709a-b1b8-4f40-b0f4-96a43a176163-kube-api-access-zxcx2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.104138 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.104078    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c155709a-b1b8-4f40-b0f4-96a43a176163-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.104208 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.104187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c155709a-b1b8-4f40-b0f4-96a43a176163-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.104241 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.104216    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c155709a-b1b8-4f40-b0f4-96a43a176163-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.104469 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.104439    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c155709a-b1b8-4f40-b0f4-96a43a176163-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.104616 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.104597    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c155709a-b1b8-4f40-b0f4-96a43a176163-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.104867 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.104844    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c155709a-b1b8-4f40-b0f4-96a43a176163-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.106421 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.106394    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c155709a-b1b8-4f40-b0f4-96a43a176163-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.106537 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.106520    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c155709a-b1b8-4f40-b0f4-96a43a176163-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.112225 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.112196    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxcx2\" (UniqueName: \"kubernetes.io/projected/c155709a-b1b8-4f40-b0f4-96a43a176163-kube-api-access-zxcx2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn\" (UID: \"c155709a-b1b8-4f40-b0f4-96a43a176163\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.178775 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.178676    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:58:50.452074 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.451800    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"]
Sep 11 13:58:50.455070 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:58:50.455039    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc155709a_b1b8_4f40_b0f4_96a43a176163.slice/crio-19c25161d1f7cb8219d1a26ef637a32023679e234ff30c5fce9284fc29a9a48c WatchSource:0}: Error finding container 19c25161d1f7cb8219d1a26ef637a32023679e234ff30c5fce9284fc29a9a48c: Status 404 returned error can't find the container with id 19c25161d1f7cb8219d1a26ef637a32023679e234ff30c5fce9284fc29a9a48c
Sep 11 13:58:50.733690 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.733567    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn" event={"ID":"c155709a-b1b8-4f40-b0f4-96a43a176163","Type":"ContainerStarted","Data":"fa9711206ce09082474d21ae43f170e5540c9c21be9187e9831546d0d16c7234"}
Sep 11 13:58:50.733690 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.733656    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn" event={"ID":"c155709a-b1b8-4f40-b0f4-96a43a176163","Type":"ContainerStarted","Data":"19c25161d1f7cb8219d1a26ef637a32023679e234ff30c5fce9284fc29a9a48c"}
Sep 11 13:58:50.752915 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:58:50.752850    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn" podStartSLOduration=1.578214027 podStartE2EDuration="1.752830371s" totalImagesPullingTime="174.616344ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:58:50.750242685 +0000 UTC m=+612.147585439" watchObservedRunningTime="2026-09-11 13:58:50.752830371 +0000 UTC m=+612.150173152"
Sep 11 13:59:00.179485 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:00.179391    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:59:00.179973 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:00.179512    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:59:00.192365 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:00.192329    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:59:00.787314 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:00.787279    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn"
Sep 11 13:59:15.372842 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.372796    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"]
Sep 11 13:59:15.376111 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.376088    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.378284 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.378260    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 11 13:59:15.384504 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.384478    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"]
Sep 11 13:59:15.421092 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.421050    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e119a128-3ef5-4c89-972a-a9c53279b31a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.421297 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.421110    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e119a128-3ef5-4c89-972a-a9c53279b31a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.421297 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.421188    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e119a128-3ef5-4c89-972a-a9c53279b31a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.421297 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.421277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmnjz\" (UniqueName: \"kubernetes.io/projected/e119a128-3ef5-4c89-972a-a9c53279b31a-kube-api-access-lmnjz\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.421599 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.421344    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e119a128-3ef5-4c89-972a-a9c53279b31a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.421599 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.421400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e119a128-3ef5-4c89-972a-a9c53279b31a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.522373 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.522331    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e119a128-3ef5-4c89-972a-a9c53279b31a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.522562 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.522383    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e119a128-3ef5-4c89-972a-a9c53279b31a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.522562 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.522406    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e119a128-3ef5-4c89-972a-a9c53279b31a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.522562 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.522452    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmnjz\" (UniqueName: \"kubernetes.io/projected/e119a128-3ef5-4c89-972a-a9c53279b31a-kube-api-access-lmnjz\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.522562 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.522476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e119a128-3ef5-4c89-972a-a9c53279b31a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.522562 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.522496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e119a128-3ef5-4c89-972a-a9c53279b31a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.522901 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.522872    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e119a128-3ef5-4c89-972a-a9c53279b31a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.523067 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.522882    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e119a128-3ef5-4c89-972a-a9c53279b31a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.523136 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.522908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e119a128-3ef5-4c89-972a-a9c53279b31a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.525117 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.525089    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e119a128-3ef5-4c89-972a-a9c53279b31a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.525296 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.525276    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e119a128-3ef5-4c89-972a-a9c53279b31a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.530453 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.530428    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmnjz\" (UniqueName: \"kubernetes.io/projected/e119a128-3ef5-4c89-972a-a9c53279b31a-kube-api-access-lmnjz\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np\" (UID: \"e119a128-3ef5-4c89-972a-a9c53279b31a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.688434 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.688339    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:15.964652 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.964603    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"]
Sep 11 13:59:15.967374 ip-10-0-128-161 kubenswrapper[2563]: W0911 13:59:15.967346    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode119a128_3ef5_4c89_972a_a9c53279b31a.slice/crio-49f8db9a08f892d79b4c685f066f39a9deff0ae30c5a5eb72cf79b91d517baa8 WatchSource:0}: Error finding container 49f8db9a08f892d79b4c685f066f39a9deff0ae30c5a5eb72cf79b91d517baa8: Status 404 returned error can't find the container with id 49f8db9a08f892d79b4c685f066f39a9deff0ae30c5a5eb72cf79b91d517baa8
Sep 11 13:59:15.969162 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:15.969142    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:59:16.853042 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:16.852999    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np" event={"ID":"e119a128-3ef5-4c89-972a-a9c53279b31a","Type":"ContainerStarted","Data":"dfef80e62b988ec5ab5a33a4bfaa0b3d74e035c869c688f69cfa43722c17d222"}
Sep 11 13:59:16.853454 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:16.853111    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np" event={"ID":"e119a128-3ef5-4c89-972a-a9c53279b31a","Type":"ContainerStarted","Data":"49f8db9a08f892d79b4c685f066f39a9deff0ae30c5a5eb72cf79b91d517baa8"}
Sep 11 13:59:16.870366 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:16.870311    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np" podStartSLOduration=1.681499903 podStartE2EDuration="1.870294049s" totalImagesPullingTime="188.794146ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:59:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:16.869331367 +0000 UTC m=+638.266674106" watchObservedRunningTime="2026-09-11 13:59:16.870294049 +0000 UTC m=+638.267636786"
Sep 11 13:59:25.688524 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:25.688474    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:25.688998 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:25.688795    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:25.701405 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:25.701372    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 13:59:25.903801 ip-10-0-128-161 kubenswrapper[2563]: I0911 13:59:25.903768    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np"
Sep 11 14:01:34.454300 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:34.454254    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"]
Sep 11 14:01:34.457791 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:34.457735    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:34.459753 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:34.459724    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-9c9fc1-serving-cert\""
Sep 11 14:01:34.468924 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:34.468891    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"]
Sep 11 14:01:34.645518 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:34.645472    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/338e2621-d789-41c8-9655-c61e206c8b8f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:34.645518 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:34.645522    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338e2621-d789-41c8-9655-c61e206c8b8f-metrics-certs\") pod \"maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:34.645794 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:34.645593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rk2f\" (UniqueName: \"kubernetes.io/projected/338e2621-d789-41c8-9655-c61e206c8b8f-kube-api-access-2rk2f\") pod \"maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:34.645794 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:34.645732    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/338e2621-d789-41c8-9655-c61e206c8b8f-maas-api-tls\") pod \"maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:34.747157 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:34.747059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/338e2621-d789-41c8-9655-c61e206c8b8f-maas-api-tls\") pod \"maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:34.747157 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:34.747099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/338e2621-d789-41c8-9655-c61e206c8b8f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:34.747157 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:34.747130    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338e2621-d789-41c8-9655-c61e206c8b8f-metrics-certs\") pod \"maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:34.747445 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:34.747173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rk2f\" (UniqueName: \"kubernetes.io/projected/338e2621-d789-41c8-9655-c61e206c8b8f-kube-api-access-2rk2f\") pod \"maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:34.747445 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:01:34.747244    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-serving-cert: secret "maas-api-e2e-inf-a-9c9fc1-serving-cert" not found
Sep 11 14:01:34.747445 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:01:34.747327    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338e2621-d789-41c8-9655-c61e206c8b8f-maas-api-tls podName:338e2621-d789-41c8-9655-c61e206c8b8f nodeName:}" failed. No retries permitted until 2026-09-11 14:01:35.247304245 +0000 UTC m=+776.644646982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/338e2621-d789-41c8-9655-c61e206c8b8f-maas-api-tls") pod "maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj" (UID: "338e2621-d789-41c8-9655-c61e206c8b8f") : secret "maas-api-e2e-inf-a-9c9fc1-serving-cert" not found
Sep 11 14:01:34.747860 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:34.747842    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/338e2621-d789-41c8-9655-c61e206c8b8f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:34.749931 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:34.749912    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338e2621-d789-41c8-9655-c61e206c8b8f-metrics-certs\") pod \"maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:34.758836 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:34.758799    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rk2f\" (UniqueName: \"kubernetes.io/projected/338e2621-d789-41c8-9655-c61e206c8b8f-kube-api-access-2rk2f\") pod \"maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:35.251595 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:35.251554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/338e2621-d789-41c8-9655-c61e206c8b8f-maas-api-tls\") pod \"maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:35.254242 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:35.254207    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/338e2621-d789-41c8-9655-c61e206c8b8f-maas-api-tls\") pod \"maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:35.371166 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:35.371116    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:35.645411 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:35.645383    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"]
Sep 11 14:01:35.647825 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:01:35.647796    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338e2621_d789_41c8_9655_c61e206c8b8f.slice/crio-fdfe600afd8d3fdd1621f8a2dd3cbf8976a17e8755532656b41cceb55ff1734a WatchSource:0}: Error finding container fdfe600afd8d3fdd1621f8a2dd3cbf8976a17e8755532656b41cceb55ff1734a: Status 404 returned error can't find the container with id fdfe600afd8d3fdd1621f8a2dd3cbf8976a17e8755532656b41cceb55ff1734a
Sep 11 14:01:36.471305 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:36.471200    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj" event={"ID":"338e2621-d789-41c8-9655-c61e206c8b8f","Type":"ContainerStarted","Data":"e142845abf5e6d8ae5c3d9eb3e975b0b600374d04bac17cb805efbfdd10e6e2a"}
Sep 11 14:01:36.471305 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:36.471251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj" event={"ID":"338e2621-d789-41c8-9655-c61e206c8b8f","Type":"ContainerStarted","Data":"fdfe600afd8d3fdd1621f8a2dd3cbf8976a17e8755532656b41cceb55ff1734a"}
Sep 11 14:01:36.471305 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:36.471285    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:36.488006 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:36.487951    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj" podStartSLOduration=2.055580578 podStartE2EDuration="2.487935131s" totalImagesPullingTime="432.354553ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:01:36.486031763 +0000 UTC m=+777.883374500" watchObservedRunningTime="2026-09-11 14:01:36.487935131 +0000 UTC m=+777.885277869"
Sep 11 14:01:41.948532 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:41.948495    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"]
Sep 11 14:01:41.952409 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:41.952381    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:41.954522 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:41.954495    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-30fbcdfa-serving-cert\""
Sep 11 14:01:41.967242 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:41.967204    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"]
Sep 11 14:01:42.018116 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:42.018070    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjvfl\" (UniqueName: \"kubernetes.io/projected/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-kube-api-access-hjvfl\") pod \"maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:42.018348 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:42.018124    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-metrics-certs\") pod \"maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:42.018348 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:42.018237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-openshift-service-ca\") pod \"maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:42.018348 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:42.018336    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-maas-api-tls\") pod \"maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:42.119846 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:42.119806    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-openshift-service-ca\") pod \"maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:42.119846 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:42.119858    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-maas-api-tls\") pod \"maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:42.120088 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:42.119909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjvfl\" (UniqueName: \"kubernetes.io/projected/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-kube-api-access-hjvfl\") pod \"maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:42.120088 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:42.119929    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-metrics-certs\") pod \"maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:42.120088 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:01:42.120040    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-serving-cert: secret "maas-api-e2e-ait-30fbcdfa-serving-cert" not found
Sep 11 14:01:42.120186 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:01:42.120147    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-maas-api-tls podName:de4e30f3-c52d-4f1f-b119-e5d283a3dea9 nodeName:}" failed. No retries permitted until 2026-09-11 14:01:42.620123641 +0000 UTC m=+784.017466361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-maas-api-tls") pod "maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7" (UID: "de4e30f3-c52d-4f1f-b119-e5d283a3dea9") : secret "maas-api-e2e-ait-30fbcdfa-serving-cert" not found
Sep 11 14:01:42.120600 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:42.120579    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-openshift-service-ca\") pod \"maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:42.122679 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:42.122657    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-metrics-certs\") pod \"maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:42.128506 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:42.128471    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjvfl\" (UniqueName: \"kubernetes.io/projected/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-kube-api-access-hjvfl\") pod \"maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:42.483258 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:42.483227    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:01:42.626547 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:42.626492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-maas-api-tls\") pod \"maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:42.629386 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:42.629351    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-maas-api-tls\") pod \"maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:42.866834 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:42.866784    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:43.006796 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.006749    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"]
Sep 11 14:01:43.011348 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.011317    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:43.013396 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.013360    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-498d98-serving-cert\""
Sep 11 14:01:43.022953 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.022551    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"]
Sep 11 14:01:43.030778 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.030747    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlqbx\" (UniqueName: \"kubernetes.io/projected/2284cb4f-b83c-4f19-871b-047e750695ef-kube-api-access-hlqbx\") pod \"maas-api-e2e-autores-498d98-798cb7dc56-7nz8b\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:43.030963 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.030790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2284cb4f-b83c-4f19-871b-047e750695ef-openshift-service-ca\") pod \"maas-api-e2e-autores-498d98-798cb7dc56-7nz8b\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:43.030963 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.030895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2284cb4f-b83c-4f19-871b-047e750695ef-maas-api-tls\") pod \"maas-api-e2e-autores-498d98-798cb7dc56-7nz8b\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:43.030963 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.030919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2284cb4f-b83c-4f19-871b-047e750695ef-metrics-certs\") pod \"maas-api-e2e-autores-498d98-798cb7dc56-7nz8b\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:43.131422 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.131323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2284cb4f-b83c-4f19-871b-047e750695ef-maas-api-tls\") pod \"maas-api-e2e-autores-498d98-798cb7dc56-7nz8b\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:43.131422 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.131354    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2284cb4f-b83c-4f19-871b-047e750695ef-metrics-certs\") pod \"maas-api-e2e-autores-498d98-798cb7dc56-7nz8b\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:43.131422 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.131394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlqbx\" (UniqueName: \"kubernetes.io/projected/2284cb4f-b83c-4f19-871b-047e750695ef-kube-api-access-hlqbx\") pod \"maas-api-e2e-autores-498d98-798cb7dc56-7nz8b\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:43.131422 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.131417    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2284cb4f-b83c-4f19-871b-047e750695ef-openshift-service-ca\") pod \"maas-api-e2e-autores-498d98-798cb7dc56-7nz8b\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:43.132089 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.132065    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2284cb4f-b83c-4f19-871b-047e750695ef-openshift-service-ca\") pod \"maas-api-e2e-autores-498d98-798cb7dc56-7nz8b\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:43.134202 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.134173    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2284cb4f-b83c-4f19-871b-047e750695ef-metrics-certs\") pod \"maas-api-e2e-autores-498d98-798cb7dc56-7nz8b\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:43.134341 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.134209    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2284cb4f-b83c-4f19-871b-047e750695ef-maas-api-tls\") pod \"maas-api-e2e-autores-498d98-798cb7dc56-7nz8b\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:43.139753 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.139719    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlqbx\" (UniqueName: \"kubernetes.io/projected/2284cb4f-b83c-4f19-871b-047e750695ef-kube-api-access-hlqbx\") pod \"maas-api-e2e-autores-498d98-798cb7dc56-7nz8b\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:43.146004 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.145965    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"]
Sep 11 14:01:43.149188 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:01:43.149157    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde4e30f3_c52d_4f1f_b119_e5d283a3dea9.slice/crio-1017e2e56ef7053dc6720568c80d1a79ed465b3e866c57c0d9e637dbb99cd07b WatchSource:0}: Error finding container 1017e2e56ef7053dc6720568c80d1a79ed465b3e866c57c0d9e637dbb99cd07b: Status 404 returned error can't find the container with id 1017e2e56ef7053dc6720568c80d1a79ed465b3e866c57c0d9e637dbb99cd07b
Sep 11 14:01:43.329607 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.329560    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:43.510358 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.510323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7" event={"ID":"de4e30f3-c52d-4f1f-b119-e5d283a3dea9","Type":"ContainerStarted","Data":"1017e2e56ef7053dc6720568c80d1a79ed465b3e866c57c0d9e637dbb99cd07b"}
Sep 11 14:01:43.607212 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:43.607175    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"]
Sep 11 14:01:43.611535 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:01:43.611500    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2284cb4f_b83c_4f19_871b_047e750695ef.slice/crio-dd3b8f9ed16464b8c6d53ef72d9e299effb662cd88eac8811aaa1f4e8a642e39 WatchSource:0}: Error finding container dd3b8f9ed16464b8c6d53ef72d9e299effb662cd88eac8811aaa1f4e8a642e39: Status 404 returned error can't find the container with id dd3b8f9ed16464b8c6d53ef72d9e299effb662cd88eac8811aaa1f4e8a642e39
Sep 11 14:01:44.517224 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:44.517173    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b" event={"ID":"2284cb4f-b83c-4f19-871b-047e750695ef","Type":"ContainerStarted","Data":"f9ffd17bff77e1fca1e15fb3f9e50eb971d2b7dc6b6395e705d60226632ca1e9"}
Sep 11 14:01:44.517736 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:44.517240    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b" event={"ID":"2284cb4f-b83c-4f19-871b-047e750695ef","Type":"ContainerStarted","Data":"dd3b8f9ed16464b8c6d53ef72d9e299effb662cd88eac8811aaa1f4e8a642e39"}
Sep 11 14:01:44.517736 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:44.517269    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:44.518883 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:44.518854    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7" event={"ID":"de4e30f3-c52d-4f1f-b119-e5d283a3dea9","Type":"ContainerStarted","Data":"c4e5d2a0397388cc9773d4a219d98f862419bd1a3958489296e4748e2c916bc2"}
Sep 11 14:01:44.518950 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:44.518916    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:01:44.533940 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:44.533882    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b" podStartSLOduration=2.166923458 podStartE2EDuration="2.533865109s" totalImagesPullingTime="366.941651ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:01:44.532247342 +0000 UTC m=+785.929590077" watchObservedRunningTime="2026-09-11 14:01:44.533865109 +0000 UTC m=+785.931207845"
Sep 11 14:01:44.552024 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:44.551971    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7" podStartSLOduration=3.076724299 podStartE2EDuration="3.551955853s" totalImagesPullingTime="475.231554ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:01:44.551118571 +0000 UTC m=+785.948461308" watchObservedRunningTime="2026-09-11 14:01:44.551955853 +0000 UTC m=+785.949298634"
Sep 11 14:01:50.529436 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:50.529401    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:01:50.530004 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:01:50.529736    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:02:03.917974 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:03.917872    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"]
Sep 11 14:02:03.921458 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:03.921433    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:03.930693 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:03.930422    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-da9534-serving-cert\""
Sep 11 14:02:03.942409 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:03.942373    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"]
Sep 11 14:02:04.026865 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:04.026828    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e0ba69-636e-4914-b7cf-3fbfb83e311d-maas-api-tls\") pod \"maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:04.027047 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:04.026881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72e0ba69-636e-4914-b7cf-3fbfb83e311d-metrics-certs\") pod \"maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:04.027047 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:04.027013    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e0ba69-636e-4914-b7cf-3fbfb83e311d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:04.027172 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:04.027067    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjkjw\" (UniqueName: \"kubernetes.io/projected/72e0ba69-636e-4914-b7cf-3fbfb83e311d-kube-api-access-qjkjw\") pod \"maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:04.128312 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:04.128270    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e0ba69-636e-4914-b7cf-3fbfb83e311d-maas-api-tls\") pod \"maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:04.128312 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:04.128327    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72e0ba69-636e-4914-b7cf-3fbfb83e311d-metrics-certs\") pod \"maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:04.128599 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:04.128386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e0ba69-636e-4914-b7cf-3fbfb83e311d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:04.128599 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:04.128419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjkjw\" (UniqueName: \"kubernetes.io/projected/72e0ba69-636e-4914-b7cf-3fbfb83e311d-kube-api-access-qjkjw\") pod \"maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:04.129236 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:04.129187    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e0ba69-636e-4914-b7cf-3fbfb83e311d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:04.131296 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:04.131272    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72e0ba69-636e-4914-b7cf-3fbfb83e311d-metrics-certs\") pod \"maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:04.131400 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:04.131311    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e0ba69-636e-4914-b7cf-3fbfb83e311d-maas-api-tls\") pod \"maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:04.139254 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:04.139225    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjkjw\" (UniqueName: \"kubernetes.io/projected/72e0ba69-636e-4914-b7cf-3fbfb83e311d-kube-api-access-qjkjw\") pod \"maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:04.240408 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:04.240320    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:04.517182 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:04.517087    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"]
Sep 11 14:02:04.521300 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:02:04.521248    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e0ba69_636e_4914_b7cf_3fbfb83e311d.slice/crio-d8600c00d50330aeb5e6b1c4e6c715a0cae54ef147732e849028c2ac3e9131f9 WatchSource:0}: Error finding container d8600c00d50330aeb5e6b1c4e6c715a0cae54ef147732e849028c2ac3e9131f9: Status 404 returned error can't find the container with id d8600c00d50330aeb5e6b1c4e6c715a0cae54ef147732e849028c2ac3e9131f9
Sep 11 14:02:04.619708 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:04.619669    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg" event={"ID":"72e0ba69-636e-4914-b7cf-3fbfb83e311d","Type":"ContainerStarted","Data":"d8600c00d50330aeb5e6b1c4e6c715a0cae54ef147732e849028c2ac3e9131f9"}
Sep 11 14:02:05.625853 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:05.625809    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg" event={"ID":"72e0ba69-636e-4914-b7cf-3fbfb83e311d","Type":"ContainerStarted","Data":"0eace4c2cc17e21765d33ef7dd96338fe3e84971db01fddbb11a8fe498b6d4cc"}
Sep 11 14:02:05.626357 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:05.625943    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:05.642667 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:05.642585    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg" podStartSLOduration=2.302140555 podStartE2EDuration="2.642567621s" totalImagesPullingTime="340.427066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:02:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:02:05.641848549 +0000 UTC m=+807.039191288" watchObservedRunningTime="2026-09-11 14:02:05.642567621 +0000 UTC m=+807.039910358"
Sep 11 14:02:11.635900 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:11.635860    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:02:12.914485 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:12.914436    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"]
Sep 11 14:02:12.914946 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:12.914739    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7" podUID="de4e30f3-c52d-4f1f-b119-e5d283a3dea9" containerName="maas-api" containerID="cri-o://c4e5d2a0397388cc9773d4a219d98f862419bd1a3958489296e4748e2c916bc2" gracePeriod=30
Sep 11 14:02:13.189213 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.189176    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:02:13.314279 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.314234    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-maas-api-tls\") pod \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") "
Sep 11 14:02:13.314489 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.314306    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-metrics-certs\") pod \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") "
Sep 11 14:02:13.314489 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.314393    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjvfl\" (UniqueName: \"kubernetes.io/projected/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-kube-api-access-hjvfl\") pod \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") "
Sep 11 14:02:13.314489 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.314436    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-openshift-service-ca\") pod \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\" (UID: \"de4e30f3-c52d-4f1f-b119-e5d283a3dea9\") "
Sep 11 14:02:13.314841 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.314807    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "de4e30f3-c52d-4f1f-b119-e5d283a3dea9" (UID: "de4e30f3-c52d-4f1f-b119-e5d283a3dea9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:02:13.316994 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.316957    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "de4e30f3-c52d-4f1f-b119-e5d283a3dea9" (UID: "de4e30f3-c52d-4f1f-b119-e5d283a3dea9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:02:13.317163 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.317142    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "de4e30f3-c52d-4f1f-b119-e5d283a3dea9" (UID: "de4e30f3-c52d-4f1f-b119-e5d283a3dea9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:02:13.317220 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.317165    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-kube-api-access-hjvfl" (OuterVolumeSpecName: "kube-api-access-hjvfl") pod "de4e30f3-c52d-4f1f-b119-e5d283a3dea9" (UID: "de4e30f3-c52d-4f1f-b119-e5d283a3dea9"). InnerVolumeSpecName "kube-api-access-hjvfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:02:13.415661 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.415553    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:02:13.415661 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.415585    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:02:13.415661 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.415597    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjvfl\" (UniqueName: \"kubernetes.io/projected/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-kube-api-access-hjvfl\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:02:13.415661 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.415606    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de4e30f3-c52d-4f1f-b119-e5d283a3dea9-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:02:13.666202 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.666104    2563 generic.go:356] "Generic (PLEG): container finished" podID="de4e30f3-c52d-4f1f-b119-e5d283a3dea9" containerID="c4e5d2a0397388cc9773d4a219d98f862419bd1a3958489296e4748e2c916bc2" exitCode=0
Sep 11 14:02:13.666202 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.666138    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"
Sep 11 14:02:13.666202 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.666178    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7" event={"ID":"de4e30f3-c52d-4f1f-b119-e5d283a3dea9","Type":"ContainerDied","Data":"c4e5d2a0397388cc9773d4a219d98f862419bd1a3958489296e4748e2c916bc2"}
Sep 11 14:02:13.666426 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.666242    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7" event={"ID":"de4e30f3-c52d-4f1f-b119-e5d283a3dea9","Type":"ContainerDied","Data":"1017e2e56ef7053dc6720568c80d1a79ed465b3e866c57c0d9e637dbb99cd07b"}
Sep 11 14:02:13.666426 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.666270    2563 scope.go:117] "RemoveContainer" containerID="c4e5d2a0397388cc9773d4a219d98f862419bd1a3958489296e4748e2c916bc2"
Sep 11 14:02:13.677048 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.677027    2563 scope.go:117] "RemoveContainer" containerID="c4e5d2a0397388cc9773d4a219d98f862419bd1a3958489296e4748e2c916bc2"
Sep 11 14:02:13.677586 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:02:13.677561    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4e5d2a0397388cc9773d4a219d98f862419bd1a3958489296e4748e2c916bc2\": container with ID starting with c4e5d2a0397388cc9773d4a219d98f862419bd1a3958489296e4748e2c916bc2 not found: ID does not exist" containerID="c4e5d2a0397388cc9773d4a219d98f862419bd1a3958489296e4748e2c916bc2"
Sep 11 14:02:13.677709 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.677596    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4e5d2a0397388cc9773d4a219d98f862419bd1a3958489296e4748e2c916bc2"} err="failed to get container status \"c4e5d2a0397388cc9773d4a219d98f862419bd1a3958489296e4748e2c916bc2\": rpc error: code = NotFound desc = could not find container \"c4e5d2a0397388cc9773d4a219d98f862419bd1a3958489296e4748e2c916bc2\": container with ID starting with c4e5d2a0397388cc9773d4a219d98f862419bd1a3958489296e4748e2c916bc2 not found: ID does not exist"
Sep 11 14:02:13.686855 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.686818    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"]
Sep 11 14:02:13.690765 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:13.690730    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-30fbcdfa-b67dd6c6-r7kz7"]
Sep 11 14:02:14.409372 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.409332    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"]
Sep 11 14:02:14.409959 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.409777    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de4e30f3-c52d-4f1f-b119-e5d283a3dea9" containerName="maas-api"
Sep 11 14:02:14.409959 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.409791    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="de4e30f3-c52d-4f1f-b119-e5d283a3dea9" containerName="maas-api"
Sep 11 14:02:14.409959 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.409854    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="de4e30f3-c52d-4f1f-b119-e5d283a3dea9" containerName="maas-api"
Sep 11 14:02:14.414385 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.414358    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.416495 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.416452    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-0d4b8af1-gw-openshift-default-dockercfg-qrc8p\""
Sep 11 14:02:14.423886 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.423852    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"]
Sep 11 14:02:14.528296 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.528245    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-token\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.528296 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.528301    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-workload-certs\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.528533 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.528359    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-data\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.528533 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.528427    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-podinfo\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.528533 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.528457    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-envoy\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.528533 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.528497    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-credential-socket\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.528716 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.528526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istiod-ca-cert\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.528716 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.528654    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnc67\" (UniqueName: \"kubernetes.io/projected/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-kube-api-access-hnc67\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.528716 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.528700    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-workload-socket\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.629795 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.629754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-envoy\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.629978 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.629811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-credential-socket\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.629978 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.629938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istiod-ca-cert\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.630111 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.630029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnc67\" (UniqueName: \"kubernetes.io/projected/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-kube-api-access-hnc67\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.630111 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.630061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-workload-socket\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.630210 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.630134    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-token\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.630210 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.630192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-credential-socket\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.630323 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.630240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-workload-certs\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.630323 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.630289    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-data\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.630434 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.630346    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-podinfo\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.630434 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.630372    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-workload-socket\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.630559 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.630543    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-workload-certs\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.630760 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.630680    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-data\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.630760 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.630688    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istiod-ca-cert\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.632428 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.632394    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-envoy\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.632995 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.632969    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-podinfo\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.637935 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.637904    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-token\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.638124 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.638101    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnc67\" (UniqueName: \"kubernetes.io/projected/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-kube-api-access-hnc67\") pod \"e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") " pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:14.727963 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:14.727866    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:15.016913 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:15.016876    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"]
Sep 11 14:02:15.019018 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:02:15.018987    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod974bb512_d4c7_47c2_8c8f_09bfa0de9cef.slice/crio-b1d51a2ea5ab0e11c9f09173cf0215a21ed072c3d1674da26779f86088708ab4 WatchSource:0}: Error finding container b1d51a2ea5ab0e11c9f09173cf0215a21ed072c3d1674da26779f86088708ab4: Status 404 returned error can't find the container with id b1d51a2ea5ab0e11c9f09173cf0215a21ed072c3d1674da26779f86088708ab4
Sep 11 14:02:15.132245 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:15.132207    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de4e30f3-c52d-4f1f-b119-e5d283a3dea9" path="/var/lib/kubelet/pods/de4e30f3-c52d-4f1f-b119-e5d283a3dea9/volumes"
Sep 11 14:02:15.679234 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:15.679202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d" event={"ID":"974bb512-d4c7-47c2-8c8f-09bfa0de9cef","Type":"ContainerStarted","Data":"b1d51a2ea5ab0e11c9f09173cf0215a21ed072c3d1674da26779f86088708ab4"}
Sep 11 14:02:18.338393 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:18.338356    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:02:18.338715 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:18.338453    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:02:18.338715 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:18.338481    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:02:18.698915 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:18.698810    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d" event={"ID":"974bb512-d4c7-47c2-8c8f-09bfa0de9cef","Type":"ContainerStarted","Data":"60602fbda2b7bd93a6d292be3c1f1c4cec4354a90acca16d42c3a6075d4f0e46"}
Sep 11 14:02:18.719817 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:18.719755    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d" podStartSLOduration=1.402872503 podStartE2EDuration="4.71973612s" totalImagesPullingTime="3.316863617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:02:18.717273325 +0000 UTC m=+820.114616064" watchObservedRunningTime="2026-09-11 14:02:18.71973612 +0000 UTC m=+820.117078857"
Sep 11 14:02:18.728684 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:18.728606    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:19.734650 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:19.734604    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:20.414037 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.414000    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"]
Sep 11 14:02:20.418090 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.418063    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:20.420246 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.420217    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-0d4b8af1-serving-cert\""
Sep 11 14:02:20.432387 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.432340    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"]
Sep 11 14:02:20.486498 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.486460    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-openshift-service-ca\") pod \"maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:20.486732 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.486563    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-maas-api-tls\") pod \"maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:20.486732 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.486584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-metrics-certs\") pod \"maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:20.486732 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.486699    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl2dl\" (UniqueName: \"kubernetes.io/projected/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-kube-api-access-jl2dl\") pod \"maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:20.588016 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.587969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-maas-api-tls\") pod \"maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:20.588016 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.588012    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-metrics-certs\") pod \"maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:20.588258 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.588047    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl2dl\" (UniqueName: \"kubernetes.io/projected/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-kube-api-access-jl2dl\") pod \"maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:20.588258 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.588086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-openshift-service-ca\") pod \"maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:20.588258 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:02:20.588150    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-serving-cert: secret "maas-api-e2e-migrate-0d4b8af1-serving-cert" not found
Sep 11 14:02:20.588258 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:02:20.588233    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-maas-api-tls podName:425b4f29-bc99-4bd9-bbf8-eb436efacf3c nodeName:}" failed. No retries permitted until 2026-09-11 14:02:21.088208788 +0000 UTC m=+822.485551509 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-maas-api-tls") pod "maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj" (UID: "425b4f29-bc99-4bd9-bbf8-eb436efacf3c") : secret "maas-api-e2e-migrate-0d4b8af1-serving-cert" not found
Sep 11 14:02:20.588892 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.588860    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-openshift-service-ca\") pod \"maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:20.590906 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.590871    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-metrics-certs\") pod \"maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:20.596572 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.596538    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl2dl\" (UniqueName: \"kubernetes.io/projected/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-kube-api-access-jl2dl\") pod \"maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:20.707475 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.707385    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:20.708552 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:20.708530    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:21.094145 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:21.094111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-maas-api-tls\") pod \"maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:21.096679 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:21.096657    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-maas-api-tls\") pod \"maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:21.330251 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:21.330207    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:21.605915 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:21.605884    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"]
Sep 11 14:02:21.608718 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:02:21.608687    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod425b4f29_bc99_4bd9_bbf8_eb436efacf3c.slice/crio-735b8cbdc56a4e03df9098df00551f2707a0602ec51f50e556f55d733b2d826a WatchSource:0}: Error finding container 735b8cbdc56a4e03df9098df00551f2707a0602ec51f50e556f55d733b2d826a: Status 404 returned error can't find the container with id 735b8cbdc56a4e03df9098df00551f2707a0602ec51f50e556f55d733b2d826a
Sep 11 14:02:21.712984 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:21.712890    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj" event={"ID":"425b4f29-bc99-4bd9-bbf8-eb436efacf3c","Type":"ContainerStarted","Data":"735b8cbdc56a4e03df9098df00551f2707a0602ec51f50e556f55d733b2d826a"}
Sep 11 14:02:22.719757 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:22.719712    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj" event={"ID":"425b4f29-bc99-4bd9-bbf8-eb436efacf3c","Type":"ContainerStarted","Data":"6e3c353ca5395820db9861b640b8f1e2adb001448324224f3ea01350c86cf0c4"}
Sep 11 14:02:22.720158 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:22.719908    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:22.735555 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:22.735497    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj" podStartSLOduration=2.167612419 podStartE2EDuration="2.735476404s" totalImagesPullingTime="567.863985ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:02:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:02:22.73511436 +0000 UTC m=+824.132457109" watchObservedRunningTime="2026-09-11 14:02:22.735476404 +0000 UTC m=+824.132819153"
Sep 11 14:02:28.735872 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:28.735843    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:02:41.825787 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:41.825748    2563 generic.go:356] "Generic (PLEG): container finished" podID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" containerID="60602fbda2b7bd93a6d292be3c1f1c4cec4354a90acca16d42c3a6075d4f0e46" exitCode=0
Sep 11 14:02:41.826334 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:41.825822    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d" event={"ID":"974bb512-d4c7-47c2-8c8f-09bfa0de9cef","Type":"ContainerDied","Data":"60602fbda2b7bd93a6d292be3c1f1c4cec4354a90acca16d42c3a6075d4f0e46"}
Sep 11 14:02:41.826411 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:41.826357    2563 scope.go:117] "RemoveContainer" containerID="60602fbda2b7bd93a6d292be3c1f1c4cec4354a90acca16d42c3a6075d4f0e46"
Sep 11 14:02:41.827518 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:41.827486    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:02:41.827642 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:41.827577    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:02:41.827710 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:41.827648    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:02:42.834208 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:42.834171    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d" event={"ID":"974bb512-d4c7-47c2-8c8f-09bfa0de9cef","Type":"ContainerStarted","Data":"768b59429385f2e0ed9d3f78fcdfea850b51623c8f5063564c5301405fe65483"}
Sep 11 14:02:43.728705 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:43.728661    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:43.728927 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:43.728902    2563 patch_prober.go:28] interesting pod/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 11 14:02:43.728987 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:43.728963    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d" podUID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 11 14:02:44.728587 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:44.728545    2563 patch_prober.go:28] interesting pod/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 11 14:02:44.729029 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:44.728603    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d" podUID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 11 14:02:45.728651 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:45.728588    2563 patch_prober.go:28] interesting pod/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 11 14:02:45.729158 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:45.728707    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d" podUID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 11 14:02:46.728597 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:46.728562    2563 patch_prober.go:28] interesting pod/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 11 14:02:46.728842 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:46.728739    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d" podUID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 11 14:02:47.728748 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:47.728704    2563 patch_prober.go:28] interesting pod/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 11 14:02:47.728936 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:47.728769    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d" podUID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 11 14:02:48.732769 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:48.732736    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:48.733205 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:48.733001    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:02:48.734259 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:02:48.734235    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:03:02.082245 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.082184    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"]
Sep 11 14:03:02.082973 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.082935    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj" podUID="425b4f29-bc99-4bd9-bbf8-eb436efacf3c" containerName="maas-api" containerID="cri-o://6e3c353ca5395820db9861b640b8f1e2adb001448324224f3ea01350c86cf0c4" gracePeriod=30
Sep 11 14:03:02.341798 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.341721    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:03:02.382270 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.382228    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-metrics-certs\") pod \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") "
Sep 11 14:03:02.382465 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.382319    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-maas-api-tls\") pod \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") "
Sep 11 14:03:02.382465 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.382348    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jl2dl\" (UniqueName: \"kubernetes.io/projected/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-kube-api-access-jl2dl\") pod \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") "
Sep 11 14:03:02.382465 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.382389    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-openshift-service-ca\") pod \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\" (UID: \"425b4f29-bc99-4bd9-bbf8-eb436efacf3c\") "
Sep 11 14:03:02.382869 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.382835    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "425b4f29-bc99-4bd9-bbf8-eb436efacf3c" (UID: "425b4f29-bc99-4bd9-bbf8-eb436efacf3c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:03:02.384769 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.384735    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-kube-api-access-jl2dl" (OuterVolumeSpecName: "kube-api-access-jl2dl") pod "425b4f29-bc99-4bd9-bbf8-eb436efacf3c" (UID: "425b4f29-bc99-4bd9-bbf8-eb436efacf3c"). InnerVolumeSpecName "kube-api-access-jl2dl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:03:02.384769 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.384756    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "425b4f29-bc99-4bd9-bbf8-eb436efacf3c" (UID: "425b4f29-bc99-4bd9-bbf8-eb436efacf3c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:03:02.384918 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.384905    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "425b4f29-bc99-4bd9-bbf8-eb436efacf3c" (UID: "425b4f29-bc99-4bd9-bbf8-eb436efacf3c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:03:02.450337 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.450297    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"]
Sep 11 14:03:02.450596 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.450545    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d" podUID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" containerName="istio-proxy" containerID="cri-o://768b59429385f2e0ed9d3f78fcdfea850b51623c8f5063564c5301405fe65483" gracePeriod=30
Sep 11 14:03:02.483579 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.483536    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:02.483579 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.483569    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:02.483579 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.483579    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jl2dl\" (UniqueName: \"kubernetes.io/projected/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-kube-api-access-jl2dl\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:02.483579 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.483589    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/425b4f29-bc99-4bd9-bbf8-eb436efacf3c-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:02.934784 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.934747    2563 generic.go:356] "Generic (PLEG): container finished" podID="425b4f29-bc99-4bd9-bbf8-eb436efacf3c" containerID="6e3c353ca5395820db9861b640b8f1e2adb001448324224f3ea01350c86cf0c4" exitCode=0
Sep 11 14:03:02.934963 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.934796    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"
Sep 11 14:03:02.934963 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.934802    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj" event={"ID":"425b4f29-bc99-4bd9-bbf8-eb436efacf3c","Type":"ContainerDied","Data":"6e3c353ca5395820db9861b640b8f1e2adb001448324224f3ea01350c86cf0c4"}
Sep 11 14:03:02.934963 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.934859    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj" event={"ID":"425b4f29-bc99-4bd9-bbf8-eb436efacf3c","Type":"ContainerDied","Data":"735b8cbdc56a4e03df9098df00551f2707a0602ec51f50e556f55d733b2d826a"}
Sep 11 14:03:02.934963 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.934883    2563 scope.go:117] "RemoveContainer" containerID="6e3c353ca5395820db9861b640b8f1e2adb001448324224f3ea01350c86cf0c4"
Sep 11 14:03:02.944410 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.944389    2563 scope.go:117] "RemoveContainer" containerID="6e3c353ca5395820db9861b640b8f1e2adb001448324224f3ea01350c86cf0c4"
Sep 11 14:03:02.944772 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:03:02.944750    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e3c353ca5395820db9861b640b8f1e2adb001448324224f3ea01350c86cf0c4\": container with ID starting with 6e3c353ca5395820db9861b640b8f1e2adb001448324224f3ea01350c86cf0c4 not found: ID does not exist" containerID="6e3c353ca5395820db9861b640b8f1e2adb001448324224f3ea01350c86cf0c4"
Sep 11 14:03:02.944833 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.944783    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e3c353ca5395820db9861b640b8f1e2adb001448324224f3ea01350c86cf0c4"} err="failed to get container status \"6e3c353ca5395820db9861b640b8f1e2adb001448324224f3ea01350c86cf0c4\": rpc error: code = NotFound desc = could not find container \"6e3c353ca5395820db9861b640b8f1e2adb001448324224f3ea01350c86cf0c4\": container with ID starting with 6e3c353ca5395820db9861b640b8f1e2adb001448324224f3ea01350c86cf0c4 not found: ID does not exist"
Sep 11 14:03:02.956439 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.956402    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"]
Sep 11 14:03:02.960299 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:02.960263    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0d4b8af1-796f7458fb-nbcgj"]
Sep 11 14:03:03.131903 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:03.131865    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="425b4f29-bc99-4bd9-bbf8-eb436efacf3c" path="/var/lib/kubelet/pods/425b4f29-bc99-4bd9-bbf8-eb436efacf3c/volumes"
Sep 11 14:03:03.735954 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:03.735909    2563 patch_prober.go:28] interesting pod/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 11 14:03:03.736141 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:03.735971    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d" podUID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 14:03:06.620283 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.620240    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"]
Sep 11 14:03:06.620860 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.620836    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="425b4f29-bc99-4bd9-bbf8-eb436efacf3c" containerName="maas-api"
Sep 11 14:03:06.620926 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.620864    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="425b4f29-bc99-4bd9-bbf8-eb436efacf3c" containerName="maas-api"
Sep 11 14:03:06.620981 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.620974    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="425b4f29-bc99-4bd9-bbf8-eb436efacf3c" containerName="maas-api"
Sep 11 14:03:06.623934 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.623853    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:06.626278 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.626247    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-bd7f15a9-serving-cert\""
Sep 11 14:03:06.634797 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.634765    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"]
Sep 11 14:03:06.720876 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.720827    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06479ca7-b3c3-42ea-bc97-286d11e837bd-openshift-service-ca\") pod \"maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:06.720876 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.720872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06479ca7-b3c3-42ea-bc97-286d11e837bd-metrics-certs\") pod \"maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:06.721095 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.720888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4q8r\" (UniqueName: \"kubernetes.io/projected/06479ca7-b3c3-42ea-bc97-286d11e837bd-kube-api-access-t4q8r\") pod \"maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:06.721095 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.721036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06479ca7-b3c3-42ea-bc97-286d11e837bd-maas-api-tls\") pod \"maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:06.821767 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.821730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06479ca7-b3c3-42ea-bc97-286d11e837bd-maas-api-tls\") pod \"maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:06.822023 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.821784    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06479ca7-b3c3-42ea-bc97-286d11e837bd-openshift-service-ca\") pod \"maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:06.822023 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.821810    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06479ca7-b3c3-42ea-bc97-286d11e837bd-metrics-certs\") pod \"maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:06.822023 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.821833    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4q8r\" (UniqueName: \"kubernetes.io/projected/06479ca7-b3c3-42ea-bc97-286d11e837bd-kube-api-access-t4q8r\") pod \"maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:06.822679 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.822594    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06479ca7-b3c3-42ea-bc97-286d11e837bd-openshift-service-ca\") pod \"maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:06.824653 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.824603    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06479ca7-b3c3-42ea-bc97-286d11e837bd-maas-api-tls\") pod \"maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:06.824756 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.824704    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06479ca7-b3c3-42ea-bc97-286d11e837bd-metrics-certs\") pod \"maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:06.829611 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.829581    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4q8r\" (UniqueName: \"kubernetes.io/projected/06479ca7-b3c3-42ea-bc97-286d11e837bd-kube-api-access-t4q8r\") pod \"maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:06.940891 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:06.940797    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:07.218779 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.218751    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"]
Sep 11 14:03:07.220972 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:03:07.220944    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06479ca7_b3c3_42ea_bc97_286d11e837bd.slice/crio-f9723e212b0769d33721647786c033f49e16add20fb0ce9f5e6cd986e6787df4 WatchSource:0}: Error finding container f9723e212b0769d33721647786c033f49e16add20fb0ce9f5e6cd986e6787df4: Status 404 returned error can't find the container with id f9723e212b0769d33721647786c033f49e16add20fb0ce9f5e6cd986e6787df4
Sep 11 14:03:07.699880 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.699850    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:03:07.730815 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.730767    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-data\") pod \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") "
Sep 11 14:03:07.730815 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.730813    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istiod-ca-cert\") pod \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") "
Sep 11 14:03:07.731091 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.730845    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnc67\" (UniqueName: \"kubernetes.io/projected/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-kube-api-access-hnc67\") pod \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") "
Sep 11 14:03:07.731091 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.730910    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-token\") pod \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") "
Sep 11 14:03:07.731091 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.730935    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-podinfo\") pod \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") "
Sep 11 14:03:07.731091 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.730952    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-workload-socket\") pod \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") "
Sep 11 14:03:07.731091 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.730998    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-workload-certs\") pod \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") "
Sep 11 14:03:07.731091 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.731039    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-envoy\") pod \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") "
Sep 11 14:03:07.731091 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.731089    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-credential-socket\") pod \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\" (UID: \"974bb512-d4c7-47c2-8c8f-09bfa0de9cef\") "
Sep 11 14:03:07.731359 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.731112    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-data" (OuterVolumeSpecName: "istio-data") pod "974bb512-d4c7-47c2-8c8f-09bfa0de9cef" (UID: "974bb512-d4c7-47c2-8c8f-09bfa0de9cef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:07.731404 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.731392    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-data\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:07.731482 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.731461    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "974bb512-d4c7-47c2-8c8f-09bfa0de9cef" (UID: "974bb512-d4c7-47c2-8c8f-09bfa0de9cef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:07.731542 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.731524    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "974bb512-d4c7-47c2-8c8f-09bfa0de9cef" (UID: "974bb512-d4c7-47c2-8c8f-09bfa0de9cef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:07.731620 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.731603    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "974bb512-d4c7-47c2-8c8f-09bfa0de9cef" (UID: "974bb512-d4c7-47c2-8c8f-09bfa0de9cef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:07.731759 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.731728    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "974bb512-d4c7-47c2-8c8f-09bfa0de9cef" (UID: "974bb512-d4c7-47c2-8c8f-09bfa0de9cef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:03:07.734258 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.734219    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-kube-api-access-hnc67" (OuterVolumeSpecName: "kube-api-access-hnc67") pod "974bb512-d4c7-47c2-8c8f-09bfa0de9cef" (UID: "974bb512-d4c7-47c2-8c8f-09bfa0de9cef"). InnerVolumeSpecName "kube-api-access-hnc67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:03:07.734411 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.734254    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "974bb512-d4c7-47c2-8c8f-09bfa0de9cef" (UID: "974bb512-d4c7-47c2-8c8f-09bfa0de9cef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:03:07.734411 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.734225    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-token" (OuterVolumeSpecName: "istio-token") pod "974bb512-d4c7-47c2-8c8f-09bfa0de9cef" (UID: "974bb512-d4c7-47c2-8c8f-09bfa0de9cef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:03:07.734411 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.734256    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "974bb512-d4c7-47c2-8c8f-09bfa0de9cef" (UID: "974bb512-d4c7-47c2-8c8f-09bfa0de9cef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:03:07.832693 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.832654    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-workload-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:07.832693 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.832687    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-envoy\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:07.832898 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.832697    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-credential-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:07.832898 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.832712    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istiod-ca-cert\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:07.832898 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.832725    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnc67\" (UniqueName: \"kubernetes.io/projected/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-kube-api-access-hnc67\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:07.832898 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.832737    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-token\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:07.832898 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.832749    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-istio-podinfo\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:07.832898 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.832760    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/974bb512-d4c7-47c2-8c8f-09bfa0de9cef-workload-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:07.962274 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.962237    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm" event={"ID":"06479ca7-b3c3-42ea-bc97-286d11e837bd","Type":"ContainerStarted","Data":"f9723e212b0769d33721647786c033f49e16add20fb0ce9f5e6cd986e6787df4"}
Sep 11 14:03:07.964149 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.964113    2563 generic.go:356] "Generic (PLEG): container finished" podID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" containerID="768b59429385f2e0ed9d3f78fcdfea850b51623c8f5063564c5301405fe65483" exitCode=0
Sep 11 14:03:07.964298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.964155    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d" event={"ID":"974bb512-d4c7-47c2-8c8f-09bfa0de9cef","Type":"ContainerDied","Data":"768b59429385f2e0ed9d3f78fcdfea850b51623c8f5063564c5301405fe65483"}
Sep 11 14:03:07.964298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.964193    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"
Sep 11 14:03:07.964298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.964207    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d" event={"ID":"974bb512-d4c7-47c2-8c8f-09bfa0de9cef","Type":"ContainerDied","Data":"b1d51a2ea5ab0e11c9f09173cf0215a21ed072c3d1674da26779f86088708ab4"}
Sep 11 14:03:07.964298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.964231    2563 scope.go:117] "RemoveContainer" containerID="768b59429385f2e0ed9d3f78fcdfea850b51623c8f5063564c5301405fe65483"
Sep 11 14:03:07.976878 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.976856    2563 scope.go:117] "RemoveContainer" containerID="60602fbda2b7bd93a6d292be3c1f1c4cec4354a90acca16d42c3a6075d4f0e46"
Sep 11 14:03:07.985756 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.985720    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"]
Sep 11 14:03:07.990712 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.990683    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-0d4b8af1-gw-openshift-default-7f6f5cdd88-kcp4d"]
Sep 11 14:03:07.994262 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.994244    2563 scope.go:117] "RemoveContainer" containerID="768b59429385f2e0ed9d3f78fcdfea850b51623c8f5063564c5301405fe65483"
Sep 11 14:03:07.994649 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:03:07.994604    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"768b59429385f2e0ed9d3f78fcdfea850b51623c8f5063564c5301405fe65483\": container with ID starting with 768b59429385f2e0ed9d3f78fcdfea850b51623c8f5063564c5301405fe65483 not found: ID does not exist" containerID="768b59429385f2e0ed9d3f78fcdfea850b51623c8f5063564c5301405fe65483"
Sep 11 14:03:07.994737 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.994660    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"768b59429385f2e0ed9d3f78fcdfea850b51623c8f5063564c5301405fe65483"} err="failed to get container status \"768b59429385f2e0ed9d3f78fcdfea850b51623c8f5063564c5301405fe65483\": rpc error: code = NotFound desc = could not find container \"768b59429385f2e0ed9d3f78fcdfea850b51623c8f5063564c5301405fe65483\": container with ID starting with 768b59429385f2e0ed9d3f78fcdfea850b51623c8f5063564c5301405fe65483 not found: ID does not exist"
Sep 11 14:03:07.994737 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.994688    2563 scope.go:117] "RemoveContainer" containerID="60602fbda2b7bd93a6d292be3c1f1c4cec4354a90acca16d42c3a6075d4f0e46"
Sep 11 14:03:07.995002 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:03:07.994981    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60602fbda2b7bd93a6d292be3c1f1c4cec4354a90acca16d42c3a6075d4f0e46\": container with ID starting with 60602fbda2b7bd93a6d292be3c1f1c4cec4354a90acca16d42c3a6075d4f0e46 not found: ID does not exist" containerID="60602fbda2b7bd93a6d292be3c1f1c4cec4354a90acca16d42c3a6075d4f0e46"
Sep 11 14:03:07.995055 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:07.995008    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60602fbda2b7bd93a6d292be3c1f1c4cec4354a90acca16d42c3a6075d4f0e46"} err="failed to get container status \"60602fbda2b7bd93a6d292be3c1f1c4cec4354a90acca16d42c3a6075d4f0e46\": rpc error: code = NotFound desc = could not find container \"60602fbda2b7bd93a6d292be3c1f1c4cec4354a90acca16d42c3a6075d4f0e46\": container with ID starting with 60602fbda2b7bd93a6d292be3c1f1c4cec4354a90acca16d42c3a6075d4f0e46 not found: ID does not exist"
Sep 11 14:03:08.973812 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:08.973769    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm" event={"ID":"06479ca7-b3c3-42ea-bc97-286d11e837bd","Type":"ContainerStarted","Data":"5d1940c898ceb5327f41e3c210072fdd9fb9fbe35158452276a34f903df4ff74"}
Sep 11 14:03:08.974230 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:08.973923    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:08.993419 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:08.993356    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm" podStartSLOduration=2.250620741 podStartE2EDuration="2.993333385s" totalImagesPullingTime="742.712644ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:03:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:03:08.99330899 +0000 UTC m=+870.390651726" watchObservedRunningTime="2026-09-11 14:03:08.993333385 +0000 UTC m=+870.390676123"
Sep 11 14:03:09.137758 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:09.137721    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" path="/var/lib/kubelet/pods/974bb512-d4c7-47c2-8c8f-09bfa0de9cef/volumes"
Sep 11 14:03:14.985434 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:14.985404    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:35.643024 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:35.642917    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"]
Sep 11 14:03:35.643680 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:35.643273    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b" podUID="2284cb4f-b83c-4f19-871b-047e750695ef" containerName="maas-api" containerID="cri-o://f9ffd17bff77e1fca1e15fb3f9e50eb971d2b7dc6b6395e705d60226632ca1e9" gracePeriod=30
Sep 11 14:03:35.908573 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:35.908506    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:03:35.995848 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:35.995807    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2284cb4f-b83c-4f19-871b-047e750695ef-maas-api-tls\") pod \"2284cb4f-b83c-4f19-871b-047e750695ef\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") "
Sep 11 14:03:35.996025 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:35.995879    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2284cb4f-b83c-4f19-871b-047e750695ef-metrics-certs\") pod \"2284cb4f-b83c-4f19-871b-047e750695ef\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") "
Sep 11 14:03:35.996025 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:35.995942    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlqbx\" (UniqueName: \"kubernetes.io/projected/2284cb4f-b83c-4f19-871b-047e750695ef-kube-api-access-hlqbx\") pod \"2284cb4f-b83c-4f19-871b-047e750695ef\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") "
Sep 11 14:03:35.996025 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:35.995973    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2284cb4f-b83c-4f19-871b-047e750695ef-openshift-service-ca\") pod \"2284cb4f-b83c-4f19-871b-047e750695ef\" (UID: \"2284cb4f-b83c-4f19-871b-047e750695ef\") "
Sep 11 14:03:35.996438 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:35.996404    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2284cb4f-b83c-4f19-871b-047e750695ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2284cb4f-b83c-4f19-871b-047e750695ef" (UID: "2284cb4f-b83c-4f19-871b-047e750695ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:03:35.998379 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:35.998351    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2284cb4f-b83c-4f19-871b-047e750695ef-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2284cb4f-b83c-4f19-871b-047e750695ef" (UID: "2284cb4f-b83c-4f19-871b-047e750695ef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:03:35.998501 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:35.998371    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2284cb4f-b83c-4f19-871b-047e750695ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2284cb4f-b83c-4f19-871b-047e750695ef" (UID: "2284cb4f-b83c-4f19-871b-047e750695ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:03:35.998557 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:35.998541    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2284cb4f-b83c-4f19-871b-047e750695ef-kube-api-access-hlqbx" (OuterVolumeSpecName: "kube-api-access-hlqbx") pod "2284cb4f-b83c-4f19-871b-047e750695ef" (UID: "2284cb4f-b83c-4f19-871b-047e750695ef"). InnerVolumeSpecName "kube-api-access-hlqbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:03:36.096732 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:36.096693    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2284cb4f-b83c-4f19-871b-047e750695ef-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:36.096917 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:36.096777    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlqbx\" (UniqueName: \"kubernetes.io/projected/2284cb4f-b83c-4f19-871b-047e750695ef-kube-api-access-hlqbx\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:36.096917 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:36.096789    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2284cb4f-b83c-4f19-871b-047e750695ef-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:36.096917 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:36.096798    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2284cb4f-b83c-4f19-871b-047e750695ef-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:36.104208 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:36.104169    2563 generic.go:356] "Generic (PLEG): container finished" podID="2284cb4f-b83c-4f19-871b-047e750695ef" containerID="f9ffd17bff77e1fca1e15fb3f9e50eb971d2b7dc6b6395e705d60226632ca1e9" exitCode=0
Sep 11 14:03:36.104369 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:36.104212    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"
Sep 11 14:03:36.104369 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:36.104243    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b" event={"ID":"2284cb4f-b83c-4f19-871b-047e750695ef","Type":"ContainerDied","Data":"f9ffd17bff77e1fca1e15fb3f9e50eb971d2b7dc6b6395e705d60226632ca1e9"}
Sep 11 14:03:36.104369 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:36.104293    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b" event={"ID":"2284cb4f-b83c-4f19-871b-047e750695ef","Type":"ContainerDied","Data":"dd3b8f9ed16464b8c6d53ef72d9e299effb662cd88eac8811aaa1f4e8a642e39"}
Sep 11 14:03:36.104369 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:36.104317    2563 scope.go:117] "RemoveContainer" containerID="f9ffd17bff77e1fca1e15fb3f9e50eb971d2b7dc6b6395e705d60226632ca1e9"
Sep 11 14:03:36.113618 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:36.113596    2563 scope.go:117] "RemoveContainer" containerID="f9ffd17bff77e1fca1e15fb3f9e50eb971d2b7dc6b6395e705d60226632ca1e9"
Sep 11 14:03:36.113996 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:03:36.113975    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9ffd17bff77e1fca1e15fb3f9e50eb971d2b7dc6b6395e705d60226632ca1e9\": container with ID starting with f9ffd17bff77e1fca1e15fb3f9e50eb971d2b7dc6b6395e705d60226632ca1e9 not found: ID does not exist" containerID="f9ffd17bff77e1fca1e15fb3f9e50eb971d2b7dc6b6395e705d60226632ca1e9"
Sep 11 14:03:36.114061 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:36.114011    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9ffd17bff77e1fca1e15fb3f9e50eb971d2b7dc6b6395e705d60226632ca1e9"} err="failed to get container status \"f9ffd17bff77e1fca1e15fb3f9e50eb971d2b7dc6b6395e705d60226632ca1e9\": rpc error: code = NotFound desc = could not find container \"f9ffd17bff77e1fca1e15fb3f9e50eb971d2b7dc6b6395e705d60226632ca1e9\": container with ID starting with f9ffd17bff77e1fca1e15fb3f9e50eb971d2b7dc6b6395e705d60226632ca1e9 not found: ID does not exist"
Sep 11 14:03:36.125149 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:36.125116    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"]
Sep 11 14:03:36.130949 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:36.130910    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-498d98-798cb7dc56-7nz8b"]
Sep 11 14:03:37.130840 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.130809    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2284cb4f-b83c-4f19-871b-047e750695ef" path="/var/lib/kubelet/pods/2284cb4f-b83c-4f19-871b-047e750695ef/volumes"
Sep 11 14:03:37.307284 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.307240    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"]
Sep 11 14:03:37.307548 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.307499    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm" podUID="06479ca7-b3c3-42ea-bc97-286d11e837bd" containerName="maas-api" containerID="cri-o://5d1940c898ceb5327f41e3c210072fdd9fb9fbe35158452276a34f903df4ff74" gracePeriod=30
Sep 11 14:03:37.564840 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.564804    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:37.710328 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.710218    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06479ca7-b3c3-42ea-bc97-286d11e837bd-metrics-certs\") pod \"06479ca7-b3c3-42ea-bc97-286d11e837bd\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") "
Sep 11 14:03:37.710328 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.710262    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4q8r\" (UniqueName: \"kubernetes.io/projected/06479ca7-b3c3-42ea-bc97-286d11e837bd-kube-api-access-t4q8r\") pod \"06479ca7-b3c3-42ea-bc97-286d11e837bd\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") "
Sep 11 14:03:37.710328 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.710316    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06479ca7-b3c3-42ea-bc97-286d11e837bd-openshift-service-ca\") pod \"06479ca7-b3c3-42ea-bc97-286d11e837bd\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") "
Sep 11 14:03:37.710616 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.710370    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06479ca7-b3c3-42ea-bc97-286d11e837bd-maas-api-tls\") pod \"06479ca7-b3c3-42ea-bc97-286d11e837bd\" (UID: \"06479ca7-b3c3-42ea-bc97-286d11e837bd\") "
Sep 11 14:03:37.710711 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.710680    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06479ca7-b3c3-42ea-bc97-286d11e837bd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "06479ca7-b3c3-42ea-bc97-286d11e837bd" (UID: "06479ca7-b3c3-42ea-bc97-286d11e837bd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:03:37.712865 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.712823    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06479ca7-b3c3-42ea-bc97-286d11e837bd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "06479ca7-b3c3-42ea-bc97-286d11e837bd" (UID: "06479ca7-b3c3-42ea-bc97-286d11e837bd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:03:37.713004 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.712895    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06479ca7-b3c3-42ea-bc97-286d11e837bd-kube-api-access-t4q8r" (OuterVolumeSpecName: "kube-api-access-t4q8r") pod "06479ca7-b3c3-42ea-bc97-286d11e837bd" (UID: "06479ca7-b3c3-42ea-bc97-286d11e837bd"). InnerVolumeSpecName "kube-api-access-t4q8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:03:37.713004 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.712950    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06479ca7-b3c3-42ea-bc97-286d11e837bd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "06479ca7-b3c3-42ea-bc97-286d11e837bd" (UID: "06479ca7-b3c3-42ea-bc97-286d11e837bd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:03:37.811259 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.811217    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06479ca7-b3c3-42ea-bc97-286d11e837bd-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:37.811259 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.811248    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06479ca7-b3c3-42ea-bc97-286d11e837bd-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:37.811259 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.811257    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4q8r\" (UniqueName: \"kubernetes.io/projected/06479ca7-b3c3-42ea-bc97-286d11e837bd-kube-api-access-t4q8r\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:37.811259 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:37.811267    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06479ca7-b3c3-42ea-bc97-286d11e837bd-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:03:38.117165 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:38.117128    2563 generic.go:356] "Generic (PLEG): container finished" podID="06479ca7-b3c3-42ea-bc97-286d11e837bd" containerID="5d1940c898ceb5327f41e3c210072fdd9fb9fbe35158452276a34f903df4ff74" exitCode=0
Sep 11 14:03:38.117353 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:38.117193    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"
Sep 11 14:03:38.117353 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:38.117221    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm" event={"ID":"06479ca7-b3c3-42ea-bc97-286d11e837bd","Type":"ContainerDied","Data":"5d1940c898ceb5327f41e3c210072fdd9fb9fbe35158452276a34f903df4ff74"}
Sep 11 14:03:38.117353 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:38.117278    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm" event={"ID":"06479ca7-b3c3-42ea-bc97-286d11e837bd","Type":"ContainerDied","Data":"f9723e212b0769d33721647786c033f49e16add20fb0ce9f5e6cd986e6787df4"}
Sep 11 14:03:38.117353 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:38.117301    2563 scope.go:117] "RemoveContainer" containerID="5d1940c898ceb5327f41e3c210072fdd9fb9fbe35158452276a34f903df4ff74"
Sep 11 14:03:38.127320 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:38.127301    2563 scope.go:117] "RemoveContainer" containerID="5d1940c898ceb5327f41e3c210072fdd9fb9fbe35158452276a34f903df4ff74"
Sep 11 14:03:38.127743 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:03:38.127715    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d1940c898ceb5327f41e3c210072fdd9fb9fbe35158452276a34f903df4ff74\": container with ID starting with 5d1940c898ceb5327f41e3c210072fdd9fb9fbe35158452276a34f903df4ff74 not found: ID does not exist" containerID="5d1940c898ceb5327f41e3c210072fdd9fb9fbe35158452276a34f903df4ff74"
Sep 11 14:03:38.127799 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:38.127758    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d1940c898ceb5327f41e3c210072fdd9fb9fbe35158452276a34f903df4ff74"} err="failed to get container status \"5d1940c898ceb5327f41e3c210072fdd9fb9fbe35158452276a34f903df4ff74\": rpc error: code = NotFound desc = could not find container \"5d1940c898ceb5327f41e3c210072fdd9fb9fbe35158452276a34f903df4ff74\": container with ID starting with 5d1940c898ceb5327f41e3c210072fdd9fb9fbe35158452276a34f903df4ff74 not found: ID does not exist"
Sep 11 14:03:38.138586 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:38.138549    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"]
Sep 11 14:03:38.143107 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:38.143074    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd7f15a9-5959b7598b-tgmlm"]
Sep 11 14:03:39.136477 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.136445    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06479ca7-b3c3-42ea-bc97-286d11e837bd" path="/var/lib/kubelet/pods/06479ca7-b3c3-42ea-bc97-286d11e837bd/volumes"
Sep 11 14:03:39.343481 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.343445    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"]
Sep 11 14:03:39.343921 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.343824    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2284cb4f-b83c-4f19-871b-047e750695ef" containerName="maas-api"
Sep 11 14:03:39.343921 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.343838    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2284cb4f-b83c-4f19-871b-047e750695ef" containerName="maas-api"
Sep 11 14:03:39.343921 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.343848    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" containerName="istio-proxy"
Sep 11 14:03:39.343921 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.343854    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" containerName="istio-proxy"
Sep 11 14:03:39.343921 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.343870    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" containerName="istio-proxy"
Sep 11 14:03:39.343921 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.343875    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" containerName="istio-proxy"
Sep 11 14:03:39.343921 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.343888    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06479ca7-b3c3-42ea-bc97-286d11e837bd" containerName="maas-api"
Sep 11 14:03:39.343921 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.343893    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="06479ca7-b3c3-42ea-bc97-286d11e837bd" containerName="maas-api"
Sep 11 14:03:39.344161 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.343943    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" containerName="istio-proxy"
Sep 11 14:03:39.344161 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.343950    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="06479ca7-b3c3-42ea-bc97-286d11e837bd" containerName="maas-api"
Sep 11 14:03:39.344161 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.343958    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2284cb4f-b83c-4f19-871b-047e750695ef" containerName="maas-api"
Sep 11 14:03:39.348723 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.348693    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.351073 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.351042    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-10d06905-openshift-default-dockercfg-qdrzv\""
Sep 11 14:03:39.361984 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.361906    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"]
Sep 11 14:03:39.525087 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.525044    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-istio-envoy\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.525087 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.525087    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28kzv\" (UniqueName: \"kubernetes.io/projected/1f174add-81da-45ba-bd01-a56b94141800-kube-api-access-28kzv\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.525331 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.525153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-workload-socket\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.525331 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.525219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f174add-81da-45ba-bd01-a56b94141800-istio-podinfo\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.525331 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.525241    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-credential-socket\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.525331 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.525261    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-istio-data\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.525331 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.525287    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f174add-81da-45ba-bd01-a56b94141800-istio-token\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.525331 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.525314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-workload-certs\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.525331 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.525329    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f174add-81da-45ba-bd01-a56b94141800-istiod-ca-cert\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.625901 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.625797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-istio-data\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.625901 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.625840    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f174add-81da-45ba-bd01-a56b94141800-istio-token\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.625901 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.625863    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-workload-certs\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.625901 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.625879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f174add-81da-45ba-bd01-a56b94141800-istiod-ca-cert\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.626349 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.625945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-istio-envoy\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.626349 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.625959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28kzv\" (UniqueName: \"kubernetes.io/projected/1f174add-81da-45ba-bd01-a56b94141800-kube-api-access-28kzv\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.626349 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.626122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-workload-socket\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.626349 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.626174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f174add-81da-45ba-bd01-a56b94141800-istio-podinfo\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.626349 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.626205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-credential-socket\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.626349 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.626282    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-istio-data\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.626349 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.626345    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-workload-certs\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.626767 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.626518    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-workload-socket\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.626767 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.626695    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-credential-socket\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.626831 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.626764    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f174add-81da-45ba-bd01-a56b94141800-istiod-ca-cert\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.628914 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.628886    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-istio-envoy\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.629034 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.629005    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f174add-81da-45ba-bd01-a56b94141800-istio-podinfo\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.634458 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.634430    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28kzv\" (UniqueName: \"kubernetes.io/projected/1f174add-81da-45ba-bd01-a56b94141800-kube-api-access-28kzv\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.634581 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.634527    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f174add-81da-45ba-bd01-a56b94141800-istio-token\") pod \"e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") " pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.661339 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.661300    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:39.963296 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.963271    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"]
Sep 11 14:03:39.965280 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:03:39.965252    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f174add_81da_45ba_bd01_a56b94141800.slice/crio-3dd5660328562cd7a0a0f843bcdbee48e8809bd5cc7ef7c5786a7ef0d9ed5efd WatchSource:0}: Error finding container 3dd5660328562cd7a0a0f843bcdbee48e8809bd5cc7ef7c5786a7ef0d9ed5efd: Status 404 returned error can't find the container with id 3dd5660328562cd7a0a0f843bcdbee48e8809bd5cc7ef7c5786a7ef0d9ed5efd
Sep 11 14:03:39.968118 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.968083    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:03:39.968227 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.968175    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:03:39.968227 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:39.968220    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:03:40.138906 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:40.138857    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2" event={"ID":"1f174add-81da-45ba-bd01-a56b94141800","Type":"ContainerStarted","Data":"ebcc6825647da461c639b4f2b3dba52a4ca2fd669ef5ef858ab1251ddb2eb77f"}
Sep 11 14:03:40.139089 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:40.138933    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2" event={"ID":"1f174add-81da-45ba-bd01-a56b94141800","Type":"ContainerStarted","Data":"3dd5660328562cd7a0a0f843bcdbee48e8809bd5cc7ef7c5786a7ef0d9ed5efd"}
Sep 11 14:03:40.159755 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:40.159695    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2" podStartSLOduration=1.159678174 podStartE2EDuration="1.159678174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:03:40.158861416 +0000 UTC m=+901.556204169" watchObservedRunningTime="2026-09-11 14:03:40.159678174 +0000 UTC m=+901.557020911"
Sep 11 14:03:40.662008 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:40.661962    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:41.667915 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:41.667860    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:42.148576 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.148545    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:42.149482 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.149458    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:03:42.628173 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.628128    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"]
Sep 11 14:03:42.628619 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.628604    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="974bb512-d4c7-47c2-8c8f-09bfa0de9cef" containerName="istio-proxy"
Sep 11 14:03:42.631920 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.631889    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:42.634296 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.634263    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-10d06905-serving-cert\""
Sep 11 14:03:42.646391 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.646353    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"]
Sep 11 14:03:42.754700 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.754644    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhxwp\" (UniqueName: \"kubernetes.io/projected/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-kube-api-access-mhxwp\") pod \"maas-api-e2e-derive-10d06905-66756f5f-wwzcf\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:42.755204 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.754719    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-metrics-certs\") pod \"maas-api-e2e-derive-10d06905-66756f5f-wwzcf\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:42.755204 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.754884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-openshift-service-ca\") pod \"maas-api-e2e-derive-10d06905-66756f5f-wwzcf\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:42.755204 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.754926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-maas-api-tls\") pod \"maas-api-e2e-derive-10d06905-66756f5f-wwzcf\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:42.856305 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.856263    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-openshift-service-ca\") pod \"maas-api-e2e-derive-10d06905-66756f5f-wwzcf\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:42.856489 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.856316    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-maas-api-tls\") pod \"maas-api-e2e-derive-10d06905-66756f5f-wwzcf\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:42.856489 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.856354    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhxwp\" (UniqueName: \"kubernetes.io/projected/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-kube-api-access-mhxwp\") pod \"maas-api-e2e-derive-10d06905-66756f5f-wwzcf\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:42.856489 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.856374    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-metrics-certs\") pod \"maas-api-e2e-derive-10d06905-66756f5f-wwzcf\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:42.857112 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.857080    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-openshift-service-ca\") pod \"maas-api-e2e-derive-10d06905-66756f5f-wwzcf\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:42.859140 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.859114    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-maas-api-tls\") pod \"maas-api-e2e-derive-10d06905-66756f5f-wwzcf\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:42.859140 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.859133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-metrics-certs\") pod \"maas-api-e2e-derive-10d06905-66756f5f-wwzcf\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:42.865784 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.865753    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhxwp\" (UniqueName: \"kubernetes.io/projected/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-kube-api-access-mhxwp\") pod \"maas-api-e2e-derive-10d06905-66756f5f-wwzcf\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:42.946877 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:42.946769    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:43.224795 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:43.224768    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"]
Sep 11 14:03:43.227004 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:03:43.226974    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0bb6dab_4d95_41ae_a8f4_7f5989e0dfb9.slice/crio-0b0b48bdcf50e48f87bff3b5af8af0c03f3b26f49975c29d43947b5cd64773de WatchSource:0}: Error finding container 0b0b48bdcf50e48f87bff3b5af8af0c03f3b26f49975c29d43947b5cd64773de: Status 404 returned error can't find the container with id 0b0b48bdcf50e48f87bff3b5af8af0c03f3b26f49975c29d43947b5cd64773de
Sep 11 14:03:43.871009 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:43.870977    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"]
Sep 11 14:03:43.876438 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:43.875665    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:43.879524 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:43.879305    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-93da06-openshift-default-dockercfg-gbrgj\""
Sep 11 14:03:43.889722 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:43.889690    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"]
Sep 11 14:03:43.967319 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:43.967275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-workload-socket\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:43.967502 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:43.967334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7l7w\" (UniqueName: \"kubernetes.io/projected/159a6029-bfb8-4d10-925a-dc8b2a2b4445-kube-api-access-p7l7w\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:43.967502 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:43.967365    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-token\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:43.967502 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:43.967407    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-envoy\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:43.967502 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:43.967470    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-podinfo\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:43.967842 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:43.967536    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istiod-ca-cert\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:43.967842 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:43.967595    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-data\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:43.967842 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:43.967721    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-credential-socket\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:43.967842 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:43.967750    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-workload-certs\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.068657 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.068596    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-workload-socket\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.068850 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.068664    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7l7w\" (UniqueName: \"kubernetes.io/projected/159a6029-bfb8-4d10-925a-dc8b2a2b4445-kube-api-access-p7l7w\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.068850 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.068688    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-token\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.068850 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.068730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-envoy\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.068850 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.068751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-podinfo\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.068850 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.068780    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istiod-ca-cert\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.068850 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.068805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-data\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.069158 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.068883    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-credential-socket\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.069158 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.068909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-workload-certs\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.069158 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.069094    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-workload-socket\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.069325 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.069303    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-credential-socket\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.069378 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.069355    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-data\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.069456 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.069434    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-workload-certs\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.069852 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.069733    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istiod-ca-cert\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.071395 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.071371    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-envoy\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.071736 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.071714    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-podinfo\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.077022 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.076983    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7l7w\" (UniqueName: \"kubernetes.io/projected/159a6029-bfb8-4d10-925a-dc8b2a2b4445-kube-api-access-p7l7w\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.078148 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.078124    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-token\") pod \"e2e-explicit-93da06-openshift-default-545686675b-s4jt4\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") " pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.162501 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.162464    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf" event={"ID":"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9","Type":"ContainerStarted","Data":"4e18c0eecbbe2df25a069ababb2c0c0918ec305f06ba837f808f08a1e29c6dc2"}
Sep 11 14:03:44.162501 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.162519    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf" event={"ID":"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9","Type":"ContainerStarted","Data":"0b0b48bdcf50e48f87bff3b5af8af0c03f3b26f49975c29d43947b5cd64773de"}
Sep 11 14:03:44.162784 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.162615    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:44.178915 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.178863    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf" podStartSLOduration=1.665449528 podStartE2EDuration="2.178843479s" totalImagesPullingTime="513.393951ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:03:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:03:44.177259489 +0000 UTC m=+905.574602225" watchObservedRunningTime="2026-09-11 14:03:44.178843479 +0000 UTC m=+905.576186258"
Sep 11 14:03:44.190775 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.190735    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:44.476019 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.475980    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"]
Sep 11 14:03:44.477096 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:03:44.477069    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod159a6029_bfb8_4d10_925a_dc8b2a2b4445.slice/crio-db23bf18ff16abfe27fff8bd1894200ac0574be3b758c7b8f97eb0fb381024b0 WatchSource:0}: Error finding container db23bf18ff16abfe27fff8bd1894200ac0574be3b758c7b8f97eb0fb381024b0: Status 404 returned error can't find the container with id db23bf18ff16abfe27fff8bd1894200ac0574be3b758c7b8f97eb0fb381024b0
Sep 11 14:03:44.479481 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.479445    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:03:44.479568 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.479532    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:03:44.479609 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:44.479571    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:03:45.168398 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:45.168357    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4" event={"ID":"159a6029-bfb8-4d10-925a-dc8b2a2b4445","Type":"ContainerStarted","Data":"f23dfabf15a893a65f94aea481f4321f240eef037fbda99b1048cae20df04950"}
Sep 11 14:03:45.168831 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:45.168412    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4" event={"ID":"159a6029-bfb8-4d10-925a-dc8b2a2b4445","Type":"ContainerStarted","Data":"db23bf18ff16abfe27fff8bd1894200ac0574be3b758c7b8f97eb0fb381024b0"}
Sep 11 14:03:45.190378 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:45.190314    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4" podStartSLOduration=2.190294789 podStartE2EDuration="2.190294789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:03:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:03:45.187927807 +0000 UTC m=+906.585270544" watchObservedRunningTime="2026-09-11 14:03:45.190294789 +0000 UTC m=+906.587637526"
Sep 11 14:03:45.191337 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:45.191305    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:45.196840 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:45.196806    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:46.173448 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:46.173414    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:46.174591 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:46.174568    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:03:48.187810 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.187776    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"]
Sep 11 14:03:48.192012 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.191984    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:48.194008 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.193983    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-93da06-serving-cert\""
Sep 11 14:03:48.204808 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.204781    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"]
Sep 11 14:03:48.309937 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.309898    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ltqk\" (UniqueName: \"kubernetes.io/projected/c4c28b82-6282-4519-821d-248ebfc86fa5-kube-api-access-2ltqk\") pod \"maas-api-e2e-explicit-93da06-64c787cc66-t5cxq\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:48.310691 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.310661    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4c28b82-6282-4519-821d-248ebfc86fa5-maas-api-tls\") pod \"maas-api-e2e-explicit-93da06-64c787cc66-t5cxq\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:48.310863 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.310773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4c28b82-6282-4519-821d-248ebfc86fa5-openshift-service-ca\") pod \"maas-api-e2e-explicit-93da06-64c787cc66-t5cxq\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:48.310863 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.310823    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c28b82-6282-4519-821d-248ebfc86fa5-metrics-certs\") pod \"maas-api-e2e-explicit-93da06-64c787cc66-t5cxq\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:48.411970 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.411923    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ltqk\" (UniqueName: \"kubernetes.io/projected/c4c28b82-6282-4519-821d-248ebfc86fa5-kube-api-access-2ltqk\") pod \"maas-api-e2e-explicit-93da06-64c787cc66-t5cxq\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:48.412182 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.411987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4c28b82-6282-4519-821d-248ebfc86fa5-maas-api-tls\") pod \"maas-api-e2e-explicit-93da06-64c787cc66-t5cxq\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:48.412182 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.412048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4c28b82-6282-4519-821d-248ebfc86fa5-openshift-service-ca\") pod \"maas-api-e2e-explicit-93da06-64c787cc66-t5cxq\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:48.412182 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.412086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c28b82-6282-4519-821d-248ebfc86fa5-metrics-certs\") pod \"maas-api-e2e-explicit-93da06-64c787cc66-t5cxq\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:48.412431 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:03:48.412188    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-serving-cert: secret "maas-api-e2e-explicit-93da06-serving-cert" not found
Sep 11 14:03:48.412431 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:03:48.412281    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4c28b82-6282-4519-821d-248ebfc86fa5-maas-api-tls podName:c4c28b82-6282-4519-821d-248ebfc86fa5 nodeName:}" failed. No retries permitted until 2026-09-11 14:03:48.91225723 +0000 UTC m=+910.309599949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c4c28b82-6282-4519-821d-248ebfc86fa5-maas-api-tls") pod "maas-api-e2e-explicit-93da06-64c787cc66-t5cxq" (UID: "c4c28b82-6282-4519-821d-248ebfc86fa5") : secret "maas-api-e2e-explicit-93da06-serving-cert" not found
Sep 11 14:03:48.412741 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.412718    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4c28b82-6282-4519-821d-248ebfc86fa5-openshift-service-ca\") pod \"maas-api-e2e-explicit-93da06-64c787cc66-t5cxq\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:48.414994 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.414965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c28b82-6282-4519-821d-248ebfc86fa5-metrics-certs\") pod \"maas-api-e2e-explicit-93da06-64c787cc66-t5cxq\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:48.420303 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.420275    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ltqk\" (UniqueName: \"kubernetes.io/projected/c4c28b82-6282-4519-821d-248ebfc86fa5-kube-api-access-2ltqk\") pod \"maas-api-e2e-explicit-93da06-64c787cc66-t5cxq\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:48.916100 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.916059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4c28b82-6282-4519-821d-248ebfc86fa5-maas-api-tls\") pod \"maas-api-e2e-explicit-93da06-64c787cc66-t5cxq\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:48.918570 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:48.918552    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4c28b82-6282-4519-821d-248ebfc86fa5-maas-api-tls\") pod \"maas-api-e2e-explicit-93da06-64c787cc66-t5cxq\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:49.109063 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:49.109026    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:49.385998 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:49.385910    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"]
Sep 11 14:03:49.389420 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:03:49.389386    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4c28b82_6282_4519_821d_248ebfc86fa5.slice/crio-d465425fc1f65cc5379ada6f4094be85ec7e2c52652c90c591109f7686b3cb42 WatchSource:0}: Error finding container d465425fc1f65cc5379ada6f4094be85ec7e2c52652c90c591109f7686b3cb42: Status 404 returned error can't find the container with id d465425fc1f65cc5379ada6f4094be85ec7e2c52652c90c591109f7686b3cb42
Sep 11 14:03:50.175080 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:50.174989    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:03:50.198699 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:50.198584    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq" event={"ID":"c4c28b82-6282-4519-821d-248ebfc86fa5","Type":"ContainerStarted","Data":"3a21b38070dbfb2d0222f442a0830bb28aa93483c4be8674265aa82db40aa2e7"}
Sep 11 14:03:50.198928 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:50.198808    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:50.198928 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:50.198858    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq" event={"ID":"c4c28b82-6282-4519-821d-248ebfc86fa5","Type":"ContainerStarted","Data":"d465425fc1f65cc5379ada6f4094be85ec7e2c52652c90c591109f7686b3cb42"}
Sep 11 14:03:50.215833 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:50.215771    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq" podStartSLOduration=1.785333126 podStartE2EDuration="2.215748717s" totalImagesPullingTime="430.415591ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:03:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:03:50.213468103 +0000 UTC m=+911.610810845" watchObservedRunningTime="2026-09-11 14:03:50.215748717 +0000 UTC m=+911.613091454"
Sep 11 14:03:56.209758 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:56.209709    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:03:58.190826 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:58.190784    2563 patch_prober.go:28] interesting pod/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 14:03:58.191220 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:03:58.190845    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2" podUID="1f174add-81da-45ba-bd01-a56b94141800" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 14:04:00.257531 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:00.257497    2563 generic.go:356] "Generic (PLEG): container finished" podID="1f174add-81da-45ba-bd01-a56b94141800" containerID="ebcc6825647da461c639b4f2b3dba52a4ca2fd669ef5ef858ab1251ddb2eb77f" exitCode=0
Sep 11 14:04:00.257989 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:00.257575    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2" event={"ID":"1f174add-81da-45ba-bd01-a56b94141800","Type":"ContainerDied","Data":"ebcc6825647da461c639b4f2b3dba52a4ca2fd669ef5ef858ab1251ddb2eb77f"}
Sep 11 14:04:00.258064 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:00.258048    2563 scope.go:117] "RemoveContainer" containerID="ebcc6825647da461c639b4f2b3dba52a4ca2fd669ef5ef858ab1251ddb2eb77f"
Sep 11 14:04:00.259267 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:00.259234    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:04:00.259331 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:00.259318    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:04:00.259370 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:00.259361    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:04:01.264224 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:01.264188    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2" event={"ID":"1f174add-81da-45ba-bd01-a56b94141800","Type":"ContainerStarted","Data":"daebcab3d16063b10952299e235d5f24843b0f1b4fa3ab0d117d7c22cc754786"}
Sep 11 14:04:01.661718 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:01.661684    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:04:01.661932 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:01.661903    2563 patch_prober.go:28] interesting pod/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 11 14:04:01.662041 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:01.661953    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2" podUID="1f174add-81da-45ba-bd01-a56b94141800" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 11 14:04:02.662670 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:02.662608    2563 patch_prober.go:28] interesting pod/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 11 14:04:02.663074 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:02.662694    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2" podUID="1f174add-81da-45ba-bd01-a56b94141800" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 11 14:04:03.662030 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:03.661992    2563 patch_prober.go:28] interesting pod/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 11 14:04:03.662221 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:03.662059    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2" podUID="1f174add-81da-45ba-bd01-a56b94141800" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 11 14:04:04.662185 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:04.662145    2563 patch_prober.go:28] interesting pod/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 11 14:04:04.662619 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:04.662202    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2" podUID="1f174add-81da-45ba-bd01-a56b94141800" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 11 14:04:05.662599 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:05.662554    2563 patch_prober.go:28] interesting pod/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 11 14:04:05.663045 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:05.662619    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2" podUID="1f174add-81da-45ba-bd01-a56b94141800" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 11 14:04:06.827566 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:06.827533    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:04:06.828278 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:06.828254    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:04:06.831447 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:06.831419    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:04:11.327118 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:11.327084    2563 generic.go:356] "Generic (PLEG): container finished" podID="159a6029-bfb8-4d10-925a-dc8b2a2b4445" containerID="f23dfabf15a893a65f94aea481f4321f240eef037fbda99b1048cae20df04950" exitCode=0
Sep 11 14:04:11.327563 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:11.327156    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4" event={"ID":"159a6029-bfb8-4d10-925a-dc8b2a2b4445","Type":"ContainerDied","Data":"f23dfabf15a893a65f94aea481f4321f240eef037fbda99b1048cae20df04950"}
Sep 11 14:04:11.327607 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:11.327589    2563 scope.go:117] "RemoveContainer" containerID="f23dfabf15a893a65f94aea481f4321f240eef037fbda99b1048cae20df04950"
Sep 11 14:04:11.328691 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:11.328655    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:04:11.328795 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:11.328733    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:04:11.328795 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:11.328764    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:04:12.333495 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:12.333456    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4" event={"ID":"159a6029-bfb8-4d10-925a-dc8b2a2b4445","Type":"ContainerStarted","Data":"29ed76818a8f97720af1d13db11db760208b2fb77bc33c4e4a247cd3b58b65e0"}
Sep 11 14:04:13.191389 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:13.191354    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:04:13.192585 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:13.192537    2563 patch_prober.go:28] interesting pod/e2e-explicit-93da06-openshift-default-545686675b-s4jt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 11 14:04:13.192795 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:13.192614    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4" podUID="159a6029-bfb8-4d10-925a-dc8b2a2b4445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 11 14:04:13.679927 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:13.678383    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"]
Sep 11 14:04:13.680456 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:13.680420    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf" podUID="c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9" containerName="maas-api" containerID="cri-o://4e18c0eecbbe2df25a069ababb2c0c0918ec305f06ba837f808f08a1e29c6dc2" gracePeriod=30
Sep 11 14:04:14.003254 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.002831    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:04:14.057268 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.057104    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"]
Sep 11 14:04:14.057477 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.057368    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2" podUID="1f174add-81da-45ba-bd01-a56b94141800" containerName="istio-proxy" containerID="cri-o://daebcab3d16063b10952299e235d5f24843b0f1b4fa3ab0d117d7c22cc754786" gracePeriod=30
Sep 11 14:04:14.104702 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.101079    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhxwp\" (UniqueName: \"kubernetes.io/projected/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-kube-api-access-mhxwp\") pod \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") "
Sep 11 14:04:14.104702 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.101229    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-maas-api-tls\") pod \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") "
Sep 11 14:04:14.104702 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.101308    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-metrics-certs\") pod \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") "
Sep 11 14:04:14.104702 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.101345    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-openshift-service-ca\") pod \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\" (UID: \"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9\") "
Sep 11 14:04:14.104702 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.102085    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9" (UID: "c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:04:14.106556 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.106492    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9" (UID: "c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:04:14.106841 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.106815    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-kube-api-access-mhxwp" (OuterVolumeSpecName: "kube-api-access-mhxwp") pod "c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9" (UID: "c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9"). InnerVolumeSpecName "kube-api-access-mhxwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:04:14.107039 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.107005    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9" (UID: "c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:04:14.193650 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.192049    2563 patch_prober.go:28] interesting pod/e2e-explicit-93da06-openshift-default-545686675b-s4jt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 11 14:04:14.193650 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.192114    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4" podUID="159a6029-bfb8-4d10-925a-dc8b2a2b4445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 11 14:04:14.206651 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.205244    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:04:14.206651 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.205286    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:04:14.206651 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.205305    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhxwp\" (UniqueName: \"kubernetes.io/projected/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-kube-api-access-mhxwp\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:04:14.206651 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.205318    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:04:14.344461 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.344423    2563 generic.go:356] "Generic (PLEG): container finished" podID="c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9" containerID="4e18c0eecbbe2df25a069ababb2c0c0918ec305f06ba837f808f08a1e29c6dc2" exitCode=0
Sep 11 14:04:14.344661 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.344473    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf" event={"ID":"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9","Type":"ContainerDied","Data":"4e18c0eecbbe2df25a069ababb2c0c0918ec305f06ba837f808f08a1e29c6dc2"}
Sep 11 14:04:14.344661 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.344516    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf" event={"ID":"c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9","Type":"ContainerDied","Data":"0b0b48bdcf50e48f87bff3b5af8af0c03f3b26f49975c29d43947b5cd64773de"}
Sep 11 14:04:14.344661 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.344538    2563 scope.go:117] "RemoveContainer" containerID="4e18c0eecbbe2df25a069ababb2c0c0918ec305f06ba837f808f08a1e29c6dc2"
Sep 11 14:04:14.344661 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.344477    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"
Sep 11 14:04:14.354270 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.354246    2563 scope.go:117] "RemoveContainer" containerID="4e18c0eecbbe2df25a069ababb2c0c0918ec305f06ba837f808f08a1e29c6dc2"
Sep 11 14:04:14.354717 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:04:14.354688    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e18c0eecbbe2df25a069ababb2c0c0918ec305f06ba837f808f08a1e29c6dc2\": container with ID starting with 4e18c0eecbbe2df25a069ababb2c0c0918ec305f06ba837f808f08a1e29c6dc2 not found: ID does not exist" containerID="4e18c0eecbbe2df25a069ababb2c0c0918ec305f06ba837f808f08a1e29c6dc2"
Sep 11 14:04:14.354814 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.354724    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e18c0eecbbe2df25a069ababb2c0c0918ec305f06ba837f808f08a1e29c6dc2"} err="failed to get container status \"4e18c0eecbbe2df25a069ababb2c0c0918ec305f06ba837f808f08a1e29c6dc2\": rpc error: code = NotFound desc = could not find container \"4e18c0eecbbe2df25a069ababb2c0c0918ec305f06ba837f808f08a1e29c6dc2\": container with ID starting with 4e18c0eecbbe2df25a069ababb2c0c0918ec305f06ba837f808f08a1e29c6dc2 not found: ID does not exist"
Sep 11 14:04:14.364229 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.364195    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"]
Sep 11 14:04:14.368097 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:14.368065    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-10d06905-66756f5f-wwzcf"]
Sep 11 14:04:15.131331 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:15.131284    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9" path="/var/lib/kubelet/pods/c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9/volumes"
Sep 11 14:04:15.192101 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:15.192050    2563 patch_prober.go:28] interesting pod/e2e-explicit-93da06-openshift-default-545686675b-s4jt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 11 14:04:15.192327 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:15.192141    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4" podUID="159a6029-bfb8-4d10-925a-dc8b2a2b4445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 11 14:04:16.174550 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:16.173660    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:04:16.191770 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:16.191730    2563 patch_prober.go:28] interesting pod/e2e-explicit-93da06-openshift-default-545686675b-s4jt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 11 14:04:16.192078 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:16.192047    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4" podUID="159a6029-bfb8-4d10-925a-dc8b2a2b4445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 11 14:04:17.192242 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:17.192189    2563 patch_prober.go:28] interesting pod/e2e-explicit-93da06-openshift-default-545686675b-s4jt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 11 14:04:17.192784 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:17.192268    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4" podUID="159a6029-bfb8-4d10-925a-dc8b2a2b4445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 11 14:04:18.195880 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:18.195846    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:04:18.197019 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:18.196999    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:04:19.330985 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.330911    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:04:19.372021 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.371986    2563 generic.go:356] "Generic (PLEG): container finished" podID="1f174add-81da-45ba-bd01-a56b94141800" containerID="daebcab3d16063b10952299e235d5f24843b0f1b4fa3ab0d117d7c22cc754786" exitCode=0
Sep 11 14:04:19.372192 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.372050    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"
Sep 11 14:04:19.372192 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.372067    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2" event={"ID":"1f174add-81da-45ba-bd01-a56b94141800","Type":"ContainerDied","Data":"daebcab3d16063b10952299e235d5f24843b0f1b4fa3ab0d117d7c22cc754786"}
Sep 11 14:04:19.372192 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.372132    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2" event={"ID":"1f174add-81da-45ba-bd01-a56b94141800","Type":"ContainerDied","Data":"3dd5660328562cd7a0a0f843bcdbee48e8809bd5cc7ef7c5786a7ef0d9ed5efd"}
Sep 11 14:04:19.372192 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.372163    2563 scope.go:117] "RemoveContainer" containerID="daebcab3d16063b10952299e235d5f24843b0f1b4fa3ab0d117d7c22cc754786"
Sep 11 14:04:19.381603 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.381576    2563 scope.go:117] "RemoveContainer" containerID="ebcc6825647da461c639b4f2b3dba52a4ca2fd669ef5ef858ab1251ddb2eb77f"
Sep 11 14:04:19.391354 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.391329    2563 scope.go:117] "RemoveContainer" containerID="daebcab3d16063b10952299e235d5f24843b0f1b4fa3ab0d117d7c22cc754786"
Sep 11 14:04:19.391755 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:04:19.391732    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daebcab3d16063b10952299e235d5f24843b0f1b4fa3ab0d117d7c22cc754786\": container with ID starting with daebcab3d16063b10952299e235d5f24843b0f1b4fa3ab0d117d7c22cc754786 not found: ID does not exist" containerID="daebcab3d16063b10952299e235d5f24843b0f1b4fa3ab0d117d7c22cc754786"
Sep 11 14:04:19.391841 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.391766    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daebcab3d16063b10952299e235d5f24843b0f1b4fa3ab0d117d7c22cc754786"} err="failed to get container status \"daebcab3d16063b10952299e235d5f24843b0f1b4fa3ab0d117d7c22cc754786\": rpc error: code = NotFound desc = could not find container \"daebcab3d16063b10952299e235d5f24843b0f1b4fa3ab0d117d7c22cc754786\": container with ID starting with daebcab3d16063b10952299e235d5f24843b0f1b4fa3ab0d117d7c22cc754786 not found: ID does not exist"
Sep 11 14:04:19.391841 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.391786    2563 scope.go:117] "RemoveContainer" containerID="ebcc6825647da461c639b4f2b3dba52a4ca2fd669ef5ef858ab1251ddb2eb77f"
Sep 11 14:04:19.392135 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:04:19.392111    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebcc6825647da461c639b4f2b3dba52a4ca2fd669ef5ef858ab1251ddb2eb77f\": container with ID starting with ebcc6825647da461c639b4f2b3dba52a4ca2fd669ef5ef858ab1251ddb2eb77f not found: ID does not exist" containerID="ebcc6825647da461c639b4f2b3dba52a4ca2fd669ef5ef858ab1251ddb2eb77f"
Sep 11 14:04:19.392186 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.392143    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebcc6825647da461c639b4f2b3dba52a4ca2fd669ef5ef858ab1251ddb2eb77f"} err="failed to get container status \"ebcc6825647da461c639b4f2b3dba52a4ca2fd669ef5ef858ab1251ddb2eb77f\": rpc error: code = NotFound desc = could not find container \"ebcc6825647da461c639b4f2b3dba52a4ca2fd669ef5ef858ab1251ddb2eb77f\": container with ID starting with ebcc6825647da461c639b4f2b3dba52a4ca2fd669ef5ef858ab1251ddb2eb77f not found: ID does not exist"
Sep 11 14:04:19.454832 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.454787    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-istio-data\") pod \"1f174add-81da-45ba-bd01-a56b94141800\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") "
Sep 11 14:04:19.454832 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.454832    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-workload-certs\") pod \"1f174add-81da-45ba-bd01-a56b94141800\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") "
Sep 11 14:04:19.455047 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.454873    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f174add-81da-45ba-bd01-a56b94141800-istio-token\") pod \"1f174add-81da-45ba-bd01-a56b94141800\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") "
Sep 11 14:04:19.455047 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.454915    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-istio-envoy\") pod \"1f174add-81da-45ba-bd01-a56b94141800\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") "
Sep 11 14:04:19.455047 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.454936    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-credential-socket\") pod \"1f174add-81da-45ba-bd01-a56b94141800\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") "
Sep 11 14:04:19.455047 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.454971    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-workload-socket\") pod \"1f174add-81da-45ba-bd01-a56b94141800\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") "
Sep 11 14:04:19.455047 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.455010    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28kzv\" (UniqueName: \"kubernetes.io/projected/1f174add-81da-45ba-bd01-a56b94141800-kube-api-access-28kzv\") pod \"1f174add-81da-45ba-bd01-a56b94141800\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") "
Sep 11 14:04:19.455293 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.455076    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f174add-81da-45ba-bd01-a56b94141800-istio-podinfo\") pod \"1f174add-81da-45ba-bd01-a56b94141800\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") "
Sep 11 14:04:19.455293 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.455111    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f174add-81da-45ba-bd01-a56b94141800-istiod-ca-cert\") pod \"1f174add-81da-45ba-bd01-a56b94141800\" (UID: \"1f174add-81da-45ba-bd01-a56b94141800\") "
Sep 11 14:04:19.455293 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.455139    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-istio-data" (OuterVolumeSpecName: "istio-data") pod "1f174add-81da-45ba-bd01-a56b94141800" (UID: "1f174add-81da-45ba-bd01-a56b94141800"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:19.455451 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.455421    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-istio-data\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:04:19.455509 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.455453    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1f174add-81da-45ba-bd01-a56b94141800" (UID: "1f174add-81da-45ba-bd01-a56b94141800"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:19.455509 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.455477    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1f174add-81da-45ba-bd01-a56b94141800" (UID: "1f174add-81da-45ba-bd01-a56b94141800"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:19.455650 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.455513    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1f174add-81da-45ba-bd01-a56b94141800" (UID: "1f174add-81da-45ba-bd01-a56b94141800"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:19.455705 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.455659    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f174add-81da-45ba-bd01-a56b94141800-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1f174add-81da-45ba-bd01-a56b94141800" (UID: "1f174add-81da-45ba-bd01-a56b94141800"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:04:19.457494 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.457465    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f174add-81da-45ba-bd01-a56b94141800-istio-token" (OuterVolumeSpecName: "istio-token") pod "1f174add-81da-45ba-bd01-a56b94141800" (UID: "1f174add-81da-45ba-bd01-a56b94141800"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:04:19.457661 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.457490    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f174add-81da-45ba-bd01-a56b94141800-kube-api-access-28kzv" (OuterVolumeSpecName: "kube-api-access-28kzv") pod "1f174add-81da-45ba-bd01-a56b94141800" (UID: "1f174add-81da-45ba-bd01-a56b94141800"). InnerVolumeSpecName "kube-api-access-28kzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:04:19.457821 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.457800    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f174add-81da-45ba-bd01-a56b94141800-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1f174add-81da-45ba-bd01-a56b94141800" (UID: "1f174add-81da-45ba-bd01-a56b94141800"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:04:19.457821 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.457789    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1f174add-81da-45ba-bd01-a56b94141800" (UID: "1f174add-81da-45ba-bd01-a56b94141800"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:19.556100 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.556066    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f174add-81da-45ba-bd01-a56b94141800-istio-token\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:04:19.556100 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.556094    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-istio-envoy\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:04:19.556100 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.556103    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-credential-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:04:19.556100 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.556113    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-workload-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:04:19.556397 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.556122    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28kzv\" (UniqueName: \"kubernetes.io/projected/1f174add-81da-45ba-bd01-a56b94141800-kube-api-access-28kzv\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:04:19.556397 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.556131    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f174add-81da-45ba-bd01-a56b94141800-istio-podinfo\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:04:19.556397 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.556140    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f174add-81da-45ba-bd01-a56b94141800-istiod-ca-cert\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:04:19.556397 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.556147    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f174add-81da-45ba-bd01-a56b94141800-workload-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:04:19.697013 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.696970    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"]
Sep 11 14:04:19.700140 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:19.700102    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-10d06905-openshift-default-568cfcccdd-q7mv2"]
Sep 11 14:04:21.131003 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:04:21.130968    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f174add-81da-45ba-bd01-a56b94141800" path="/var/lib/kubelet/pods/1f174add-81da-45ba-bd01-a56b94141800/volumes"
Sep 11 14:05:18.176435 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.176385    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"]
Sep 11 14:05:18.177080 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.176691    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq" podUID="c4c28b82-6282-4519-821d-248ebfc86fa5" containerName="maas-api" containerID="cri-o://3a21b38070dbfb2d0222f442a0830bb28aa93483c4be8674265aa82db40aa2e7" gracePeriod=30
Sep 11 14:05:18.434222 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.434148    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:05:18.482706 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.482669    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c28b82-6282-4519-821d-248ebfc86fa5-metrics-certs\") pod \"c4c28b82-6282-4519-821d-248ebfc86fa5\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") "
Sep 11 14:05:18.482706 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.482709    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4c28b82-6282-4519-821d-248ebfc86fa5-maas-api-tls\") pod \"c4c28b82-6282-4519-821d-248ebfc86fa5\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") "
Sep 11 14:05:18.482964 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.482766    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4c28b82-6282-4519-821d-248ebfc86fa5-openshift-service-ca\") pod \"c4c28b82-6282-4519-821d-248ebfc86fa5\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") "
Sep 11 14:05:18.482964 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.482796    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ltqk\" (UniqueName: \"kubernetes.io/projected/c4c28b82-6282-4519-821d-248ebfc86fa5-kube-api-access-2ltqk\") pod \"c4c28b82-6282-4519-821d-248ebfc86fa5\" (UID: \"c4c28b82-6282-4519-821d-248ebfc86fa5\") "
Sep 11 14:05:18.483183 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.483153    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4c28b82-6282-4519-821d-248ebfc86fa5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c4c28b82-6282-4519-821d-248ebfc86fa5" (UID: "c4c28b82-6282-4519-821d-248ebfc86fa5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:05:18.485096 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.485071    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4c28b82-6282-4519-821d-248ebfc86fa5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c4c28b82-6282-4519-821d-248ebfc86fa5" (UID: "c4c28b82-6282-4519-821d-248ebfc86fa5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:05:18.485218 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.485135    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4c28b82-6282-4519-821d-248ebfc86fa5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c4c28b82-6282-4519-821d-248ebfc86fa5" (UID: "c4c28b82-6282-4519-821d-248ebfc86fa5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:05:18.485218 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.485150    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4c28b82-6282-4519-821d-248ebfc86fa5-kube-api-access-2ltqk" (OuterVolumeSpecName: "kube-api-access-2ltqk") pod "c4c28b82-6282-4519-821d-248ebfc86fa5" (UID: "c4c28b82-6282-4519-821d-248ebfc86fa5"). InnerVolumeSpecName "kube-api-access-2ltqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:05:18.584031 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.583991    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4c28b82-6282-4519-821d-248ebfc86fa5-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:18.584031 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.584033    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ltqk\" (UniqueName: \"kubernetes.io/projected/c4c28b82-6282-4519-821d-248ebfc86fa5-kube-api-access-2ltqk\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:18.584245 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.584048    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4c28b82-6282-4519-821d-248ebfc86fa5-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:18.584245 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.584063    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4c28b82-6282-4519-821d-248ebfc86fa5-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:18.667484 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.667451    2563 generic.go:356] "Generic (PLEG): container finished" podID="c4c28b82-6282-4519-821d-248ebfc86fa5" containerID="3a21b38070dbfb2d0222f442a0830bb28aa93483c4be8674265aa82db40aa2e7" exitCode=0
Sep 11 14:05:18.667729 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.667496    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq" event={"ID":"c4c28b82-6282-4519-821d-248ebfc86fa5","Type":"ContainerDied","Data":"3a21b38070dbfb2d0222f442a0830bb28aa93483c4be8674265aa82db40aa2e7"}
Sep 11 14:05:18.667729 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.667505    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"
Sep 11 14:05:18.667729 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.667548    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq" event={"ID":"c4c28b82-6282-4519-821d-248ebfc86fa5","Type":"ContainerDied","Data":"d465425fc1f65cc5379ada6f4094be85ec7e2c52652c90c591109f7686b3cb42"}
Sep 11 14:05:18.667729 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.667576    2563 scope.go:117] "RemoveContainer" containerID="3a21b38070dbfb2d0222f442a0830bb28aa93483c4be8674265aa82db40aa2e7"
Sep 11 14:05:18.677601 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.677576    2563 scope.go:117] "RemoveContainer" containerID="3a21b38070dbfb2d0222f442a0830bb28aa93483c4be8674265aa82db40aa2e7"
Sep 11 14:05:18.678050 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:05:18.678024    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a21b38070dbfb2d0222f442a0830bb28aa93483c4be8674265aa82db40aa2e7\": container with ID starting with 3a21b38070dbfb2d0222f442a0830bb28aa93483c4be8674265aa82db40aa2e7 not found: ID does not exist" containerID="3a21b38070dbfb2d0222f442a0830bb28aa93483c4be8674265aa82db40aa2e7"
Sep 11 14:05:18.678165 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.678065    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a21b38070dbfb2d0222f442a0830bb28aa93483c4be8674265aa82db40aa2e7"} err="failed to get container status \"3a21b38070dbfb2d0222f442a0830bb28aa93483c4be8674265aa82db40aa2e7\": rpc error: code = NotFound desc = could not find container \"3a21b38070dbfb2d0222f442a0830bb28aa93483c4be8674265aa82db40aa2e7\": container with ID starting with 3a21b38070dbfb2d0222f442a0830bb28aa93483c4be8674265aa82db40aa2e7 not found: ID does not exist"
Sep 11 14:05:18.688319 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.688231    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"]
Sep 11 14:05:18.691917 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:18.691881    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-93da06-64c787cc66-t5cxq"]
Sep 11 14:05:19.131397 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:19.131361    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4c28b82-6282-4519-821d-248ebfc86fa5" path="/var/lib/kubelet/pods/c4c28b82-6282-4519-821d-248ebfc86fa5/volumes"
Sep 11 14:05:25.136741 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.136702    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"]
Sep 11 14:05:25.137224 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.136935    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4" podUID="159a6029-bfb8-4d10-925a-dc8b2a2b4445" containerName="istio-proxy" containerID="cri-o://29ed76818a8f97720af1d13db11db760208b2fb77bc33c4e4a247cd3b58b65e0" gracePeriod=30
Sep 11 14:05:25.872247 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.872207    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"]
Sep 11 14:05:25.872693 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.872674    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f174add-81da-45ba-bd01-a56b94141800" containerName="istio-proxy"
Sep 11 14:05:25.872693 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.872694    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f174add-81da-45ba-bd01-a56b94141800" containerName="istio-proxy"
Sep 11 14:05:25.872882 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.872710    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9" containerName="maas-api"
Sep 11 14:05:25.872882 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.872719    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9" containerName="maas-api"
Sep 11 14:05:25.872882 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.872728    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f174add-81da-45ba-bd01-a56b94141800" containerName="istio-proxy"
Sep 11 14:05:25.872882 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.872738    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f174add-81da-45ba-bd01-a56b94141800" containerName="istio-proxy"
Sep 11 14:05:25.872882 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.872760    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4c28b82-6282-4519-821d-248ebfc86fa5" containerName="maas-api"
Sep 11 14:05:25.872882 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.872768    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4c28b82-6282-4519-821d-248ebfc86fa5" containerName="maas-api"
Sep 11 14:05:25.872882 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.872869    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f174add-81da-45ba-bd01-a56b94141800" containerName="istio-proxy"
Sep 11 14:05:25.872882 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.872880    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0bb6dab-4d95-41ae-a8f4-7f5989e0dfb9" containerName="maas-api"
Sep 11 14:05:25.873258 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.872891    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4c28b82-6282-4519-821d-248ebfc86fa5" containerName="maas-api"
Sep 11 14:05:25.876515 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.876485    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:25.879939 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.879697    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-4edac4-openshift-default-dockercfg-6wd8h\""
Sep 11 14:05:25.890008 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.889951    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"]
Sep 11 14:05:25.937709 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.937667    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-workload-certs\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:25.937709 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.937715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-podinfo\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:25.937946 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.937784    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-token\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:25.937946 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.937830    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k699q\" (UniqueName: \"kubernetes.io/projected/bda4e888-9a66-4a7a-bb7c-ef69bd834079-kube-api-access-k699q\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:25.937946 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.937855    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-data\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:25.937946 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.937927    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-envoy\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:25.938073 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.937965    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istiod-ca-cert\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:25.938073 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.937992    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-credential-socket\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:25.938073 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:25.938051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-workload-socket\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.038842 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.038800    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-envoy\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.039035 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.038856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istiod-ca-cert\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.039035 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.038895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-credential-socket\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.039035 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.038927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-workload-socket\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.039035 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.038984    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-workload-certs\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.039234 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.039102    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-podinfo\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.039234 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.039153    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-token\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.039234 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.039207    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k699q\" (UniqueName: \"kubernetes.io/projected/bda4e888-9a66-4a7a-bb7c-ef69bd834079-kube-api-access-k699q\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.039392 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.039237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-data\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.039392 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.039349    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-credential-socket\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.039392 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.039364    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-workload-certs\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.039810 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.039724    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-data\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.039959 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.039864    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-workload-socket\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.040032 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.040014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istiod-ca-cert\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.041480 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.041454    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-envoy\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.041797 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.041778    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-podinfo\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.047847 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.047815    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-token\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.047984 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.047930    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k699q\" (UniqueName: \"kubernetes.io/projected/bda4e888-9a66-4a7a-bb7c-ef69bd834079-kube-api-access-k699q\") pod \"e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") " pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.192029 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.191940    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:26.473640 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.473396    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"]
Sep 11 14:05:26.476782 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:05:26.476745    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda4e888_9a66_4a7a_bb7c_ef69bd834079.slice/crio-ea661dc232f5938f1d5c537b8da857bad99511f6d3721ad65df7617f24937d58 WatchSource:0}: Error finding container ea661dc232f5938f1d5c537b8da857bad99511f6d3721ad65df7617f24937d58: Status 404 returned error can't find the container with id ea661dc232f5938f1d5c537b8da857bad99511f6d3721ad65df7617f24937d58
Sep 11 14:05:26.479442 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.479397    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:05:26.479533 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.479498    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:05:26.479572 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.479540    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:05:26.709276 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.709207    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl" event={"ID":"bda4e888-9a66-4a7a-bb7c-ef69bd834079","Type":"ContainerStarted","Data":"119b73b0a805a66b1fbb0f8b9d2ec2eddb2b763613e0c62358350bc3b4848d5c"}
Sep 11 14:05:26.709475 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.709304    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl" event={"ID":"bda4e888-9a66-4a7a-bb7c-ef69bd834079","Type":"ContainerStarted","Data":"ea661dc232f5938f1d5c537b8da857bad99511f6d3721ad65df7617f24937d58"}
Sep 11 14:05:26.730173 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:26.730048    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl" podStartSLOduration=1.7300244409999999 podStartE2EDuration="1.730024441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:05:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:05:26.727317118 +0000 UTC m=+1008.124659855" watchObservedRunningTime="2026-09-11 14:05:26.730024441 +0000 UTC m=+1008.127367180"
Sep 11 14:05:27.192364 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:27.192323    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:27.194032 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:27.193998    2563 patch_prober.go:28] interesting pod/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 11 14:05:27.194165 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:27.194066    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl" podUID="bda4e888-9a66-4a7a-bb7c-ef69bd834079" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 11 14:05:28.197024 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:28.196994    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:28.727384 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:28.727347    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:28.728550 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:28.728527    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:30.403366 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.403296    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:05:30.475751 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.475708    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-token\") pod \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") "
Sep 11 14:05:30.475924 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.475766    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-podinfo\") pod \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") "
Sep 11 14:05:30.475924 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.475809    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-workload-certs\") pod \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") "
Sep 11 14:05:30.475924 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.475849    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-credential-socket\") pod \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") "
Sep 11 14:05:30.475924 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.475882    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istiod-ca-cert\") pod \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") "
Sep 11 14:05:30.476154 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.475927    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-data\") pod \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") "
Sep 11 14:05:30.476154 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.475960    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7l7w\" (UniqueName: \"kubernetes.io/projected/159a6029-bfb8-4d10-925a-dc8b2a2b4445-kube-api-access-p7l7w\") pod \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") "
Sep 11 14:05:30.476154 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.475983    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-workload-socket\") pod \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") "
Sep 11 14:05:30.476154 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.476014    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-envoy\") pod \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\" (UID: \"159a6029-bfb8-4d10-925a-dc8b2a2b4445\") "
Sep 11 14:05:30.476415 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.476124    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "159a6029-bfb8-4d10-925a-dc8b2a2b4445" (UID: "159a6029-bfb8-4d10-925a-dc8b2a2b4445"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:30.476415 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.476210    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "159a6029-bfb8-4d10-925a-dc8b2a2b4445" (UID: "159a6029-bfb8-4d10-925a-dc8b2a2b4445"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:30.476415 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.476331    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-workload-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:30.476415 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.476373    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-credential-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:30.476415 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.476405    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "159a6029-bfb8-4d10-925a-dc8b2a2b4445" (UID: "159a6029-bfb8-4d10-925a-dc8b2a2b4445"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:30.476874 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.476743    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-data" (OuterVolumeSpecName: "istio-data") pod "159a6029-bfb8-4d10-925a-dc8b2a2b4445" (UID: "159a6029-bfb8-4d10-925a-dc8b2a2b4445"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:30.477011 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.476986    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "159a6029-bfb8-4d10-925a-dc8b2a2b4445" (UID: "159a6029-bfb8-4d10-925a-dc8b2a2b4445"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:05:30.478475 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.478434    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "159a6029-bfb8-4d10-925a-dc8b2a2b4445" (UID: "159a6029-bfb8-4d10-925a-dc8b2a2b4445"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:30.478602 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.478479    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-token" (OuterVolumeSpecName: "istio-token") pod "159a6029-bfb8-4d10-925a-dc8b2a2b4445" (UID: "159a6029-bfb8-4d10-925a-dc8b2a2b4445"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:05:30.478733 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.478710    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "159a6029-bfb8-4d10-925a-dc8b2a2b4445" (UID: "159a6029-bfb8-4d10-925a-dc8b2a2b4445"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:05:30.478936 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.478913    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/159a6029-bfb8-4d10-925a-dc8b2a2b4445-kube-api-access-p7l7w" (OuterVolumeSpecName: "kube-api-access-p7l7w") pod "159a6029-bfb8-4d10-925a-dc8b2a2b4445" (UID: "159a6029-bfb8-4d10-925a-dc8b2a2b4445"). InnerVolumeSpecName "kube-api-access-p7l7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:05:30.577037 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.576997    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-envoy\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:30.577037 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.577031    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-token\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:30.577037 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.577040    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-podinfo\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:30.577037 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.577048    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istiod-ca-cert\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:30.577334 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.577057    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-istio-data\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:30.577334 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.577065    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7l7w\" (UniqueName: \"kubernetes.io/projected/159a6029-bfb8-4d10-925a-dc8b2a2b4445-kube-api-access-p7l7w\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:30.577334 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.577074    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/159a6029-bfb8-4d10-925a-dc8b2a2b4445-workload-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:30.738490 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.738399    2563 generic.go:356] "Generic (PLEG): container finished" podID="159a6029-bfb8-4d10-925a-dc8b2a2b4445" containerID="29ed76818a8f97720af1d13db11db760208b2fb77bc33c4e4a247cd3b58b65e0" exitCode=0
Sep 11 14:05:30.738490 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.738451    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"
Sep 11 14:05:30.738711 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.738490    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4" event={"ID":"159a6029-bfb8-4d10-925a-dc8b2a2b4445","Type":"ContainerDied","Data":"29ed76818a8f97720af1d13db11db760208b2fb77bc33c4e4a247cd3b58b65e0"}
Sep 11 14:05:30.738711 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.738543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4" event={"ID":"159a6029-bfb8-4d10-925a-dc8b2a2b4445","Type":"ContainerDied","Data":"db23bf18ff16abfe27fff8bd1894200ac0574be3b758c7b8f97eb0fb381024b0"}
Sep 11 14:05:30.738711 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.738571    2563 scope.go:117] "RemoveContainer" containerID="29ed76818a8f97720af1d13db11db760208b2fb77bc33c4e4a247cd3b58b65e0"
Sep 11 14:05:30.748005 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.747983    2563 scope.go:117] "RemoveContainer" containerID="f23dfabf15a893a65f94aea481f4321f240eef037fbda99b1048cae20df04950"
Sep 11 14:05:30.756779 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.756736    2563 scope.go:117] "RemoveContainer" containerID="29ed76818a8f97720af1d13db11db760208b2fb77bc33c4e4a247cd3b58b65e0"
Sep 11 14:05:30.757197 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:05:30.757163    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29ed76818a8f97720af1d13db11db760208b2fb77bc33c4e4a247cd3b58b65e0\": container with ID starting with 29ed76818a8f97720af1d13db11db760208b2fb77bc33c4e4a247cd3b58b65e0 not found: ID does not exist" containerID="29ed76818a8f97720af1d13db11db760208b2fb77bc33c4e4a247cd3b58b65e0"
Sep 11 14:05:30.757272 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.757210    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29ed76818a8f97720af1d13db11db760208b2fb77bc33c4e4a247cd3b58b65e0"} err="failed to get container status \"29ed76818a8f97720af1d13db11db760208b2fb77bc33c4e4a247cd3b58b65e0\": rpc error: code = NotFound desc = could not find container \"29ed76818a8f97720af1d13db11db760208b2fb77bc33c4e4a247cd3b58b65e0\": container with ID starting with 29ed76818a8f97720af1d13db11db760208b2fb77bc33c4e4a247cd3b58b65e0 not found: ID does not exist"
Sep 11 14:05:30.757272 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.757237    2563 scope.go:117] "RemoveContainer" containerID="f23dfabf15a893a65f94aea481f4321f240eef037fbda99b1048cae20df04950"
Sep 11 14:05:30.757568 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:05:30.757545    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f23dfabf15a893a65f94aea481f4321f240eef037fbda99b1048cae20df04950\": container with ID starting with f23dfabf15a893a65f94aea481f4321f240eef037fbda99b1048cae20df04950 not found: ID does not exist" containerID="f23dfabf15a893a65f94aea481f4321f240eef037fbda99b1048cae20df04950"
Sep 11 14:05:30.757729 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.757576    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f23dfabf15a893a65f94aea481f4321f240eef037fbda99b1048cae20df04950"} err="failed to get container status \"f23dfabf15a893a65f94aea481f4321f240eef037fbda99b1048cae20df04950\": rpc error: code = NotFound desc = could not find container \"f23dfabf15a893a65f94aea481f4321f240eef037fbda99b1048cae20df04950\": container with ID starting with f23dfabf15a893a65f94aea481f4321f240eef037fbda99b1048cae20df04950 not found: ID does not exist"
Sep 11 14:05:30.758764 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.758744    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"]
Sep 11 14:05:30.764513 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:30.764486    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-93da06-openshift-default-545686675b-s4jt4"]
Sep 11 14:05:31.133883 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.133848    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="159a6029-bfb8-4d10-925a-dc8b2a2b4445" path="/var/lib/kubelet/pods/159a6029-bfb8-4d10-925a-dc8b2a2b4445/volumes"
Sep 11 14:05:31.434947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.434853    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"]
Sep 11 14:05:31.435413 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.435097    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj" podUID="338e2621-d789-41c8-9655-c61e206c8b8f" containerName="maas-api" containerID="cri-o://e142845abf5e6d8ae5c3d9eb3e975b0b600374d04bac17cb805efbfdd10e6e2a" gracePeriod=30
Sep 11 14:05:31.680426 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.680395    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:05:31.746306 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.746215    2563 generic.go:356] "Generic (PLEG): container finished" podID="338e2621-d789-41c8-9655-c61e206c8b8f" containerID="e142845abf5e6d8ae5c3d9eb3e975b0b600374d04bac17cb805efbfdd10e6e2a" exitCode=0
Sep 11 14:05:31.746306 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.746265    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"
Sep 11 14:05:31.746502 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.746263    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj" event={"ID":"338e2621-d789-41c8-9655-c61e206c8b8f","Type":"ContainerDied","Data":"e142845abf5e6d8ae5c3d9eb3e975b0b600374d04bac17cb805efbfdd10e6e2a"}
Sep 11 14:05:31.746502 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.746376    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj" event={"ID":"338e2621-d789-41c8-9655-c61e206c8b8f","Type":"ContainerDied","Data":"fdfe600afd8d3fdd1621f8a2dd3cbf8976a17e8755532656b41cceb55ff1734a"}
Sep 11 14:05:31.746502 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.746399    2563 scope.go:117] "RemoveContainer" containerID="e142845abf5e6d8ae5c3d9eb3e975b0b600374d04bac17cb805efbfdd10e6e2a"
Sep 11 14:05:31.755535 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.755513    2563 scope.go:117] "RemoveContainer" containerID="e142845abf5e6d8ae5c3d9eb3e975b0b600374d04bac17cb805efbfdd10e6e2a"
Sep 11 14:05:31.755996 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:05:31.755975    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e142845abf5e6d8ae5c3d9eb3e975b0b600374d04bac17cb805efbfdd10e6e2a\": container with ID starting with e142845abf5e6d8ae5c3d9eb3e975b0b600374d04bac17cb805efbfdd10e6e2a not found: ID does not exist" containerID="e142845abf5e6d8ae5c3d9eb3e975b0b600374d04bac17cb805efbfdd10e6e2a"
Sep 11 14:05:31.756076 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.756010    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e142845abf5e6d8ae5c3d9eb3e975b0b600374d04bac17cb805efbfdd10e6e2a"} err="failed to get container status \"e142845abf5e6d8ae5c3d9eb3e975b0b600374d04bac17cb805efbfdd10e6e2a\": rpc error: code = NotFound desc = could not find container \"e142845abf5e6d8ae5c3d9eb3e975b0b600374d04bac17cb805efbfdd10e6e2a\": container with ID starting with e142845abf5e6d8ae5c3d9eb3e975b0b600374d04bac17cb805efbfdd10e6e2a not found: ID does not exist"
Sep 11 14:05:31.786133 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.786096    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/338e2621-d789-41c8-9655-c61e206c8b8f-openshift-service-ca\") pod \"338e2621-d789-41c8-9655-c61e206c8b8f\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") "
Sep 11 14:05:31.786300 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.786165    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/338e2621-d789-41c8-9655-c61e206c8b8f-maas-api-tls\") pod \"338e2621-d789-41c8-9655-c61e206c8b8f\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") "
Sep 11 14:05:31.786300 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.786196    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338e2621-d789-41c8-9655-c61e206c8b8f-metrics-certs\") pod \"338e2621-d789-41c8-9655-c61e206c8b8f\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") "
Sep 11 14:05:31.786414 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.786296    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rk2f\" (UniqueName: \"kubernetes.io/projected/338e2621-d789-41c8-9655-c61e206c8b8f-kube-api-access-2rk2f\") pod \"338e2621-d789-41c8-9655-c61e206c8b8f\" (UID: \"338e2621-d789-41c8-9655-c61e206c8b8f\") "
Sep 11 14:05:31.786566 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.786543    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/338e2621-d789-41c8-9655-c61e206c8b8f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "338e2621-d789-41c8-9655-c61e206c8b8f" (UID: "338e2621-d789-41c8-9655-c61e206c8b8f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:05:31.788683 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.788651    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/338e2621-d789-41c8-9655-c61e206c8b8f-kube-api-access-2rk2f" (OuterVolumeSpecName: "kube-api-access-2rk2f") pod "338e2621-d789-41c8-9655-c61e206c8b8f" (UID: "338e2621-d789-41c8-9655-c61e206c8b8f"). InnerVolumeSpecName "kube-api-access-2rk2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:05:31.788683 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.788664    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/338e2621-d789-41c8-9655-c61e206c8b8f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "338e2621-d789-41c8-9655-c61e206c8b8f" (UID: "338e2621-d789-41c8-9655-c61e206c8b8f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:05:31.788840 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.788656    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/338e2621-d789-41c8-9655-c61e206c8b8f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "338e2621-d789-41c8-9655-c61e206c8b8f" (UID: "338e2621-d789-41c8-9655-c61e206c8b8f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:05:31.887302 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.887256    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rk2f\" (UniqueName: \"kubernetes.io/projected/338e2621-d789-41c8-9655-c61e206c8b8f-kube-api-access-2rk2f\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:31.887302 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.887292    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/338e2621-d789-41c8-9655-c61e206c8b8f-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:31.887302 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.887302    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/338e2621-d789-41c8-9655-c61e206c8b8f-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:31.887302 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:31.887310    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338e2621-d789-41c8-9655-c61e206c8b8f-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:32.066827 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:32.066793    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"]
Sep 11 14:05:32.074599 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:32.074560    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c9fc1-d98d7845c-kz2tj"]
Sep 11 14:05:33.131520 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:33.131488    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="338e2621-d789-41c8-9655-c61e206c8b8f" path="/var/lib/kubelet/pods/338e2621-d789-41c8-9655-c61e206c8b8f/volumes"
Sep 11 14:05:39.425699 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:39.425588    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"]
Sep 11 14:05:39.426161 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:39.425830    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl" podUID="bda4e888-9a66-4a7a-bb7c-ef69bd834079" containerName="istio-proxy" containerID="cri-o://119b73b0a805a66b1fbb0f8b9d2ec2eddb2b763613e0c62358350bc3b4848d5c" gracePeriod=30
Sep 11 14:05:43.729969 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:43.729927    2563 patch_prober.go:28] interesting pod/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 14:05:43.730380 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:43.730007    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl" podUID="bda4e888-9a66-4a7a-bb7c-ef69bd834079" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 14:05:44.683397 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.683323    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:44.802260 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.802217    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istiod-ca-cert\") pod \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") "
Sep 11 14:05:44.802260 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.802267    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k699q\" (UniqueName: \"kubernetes.io/projected/bda4e888-9a66-4a7a-bb7c-ef69bd834079-kube-api-access-k699q\") pod \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") "
Sep 11 14:05:44.802822 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.802298    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-token\") pod \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") "
Sep 11 14:05:44.802822 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.802320    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-envoy\") pod \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") "
Sep 11 14:05:44.802822 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.802351    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-workload-certs\") pod \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") "
Sep 11 14:05:44.802822 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.802400    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-workload-socket\") pod \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") "
Sep 11 14:05:44.802822 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.802459    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-data\") pod \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") "
Sep 11 14:05:44.802822 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.802531    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-podinfo\") pod \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") "
Sep 11 14:05:44.802822 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.802561    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-credential-socket\") pod \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\" (UID: \"bda4e888-9a66-4a7a-bb7c-ef69bd834079\") "
Sep 11 14:05:44.802822 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.802711    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bda4e888-9a66-4a7a-bb7c-ef69bd834079" (UID: "bda4e888-9a66-4a7a-bb7c-ef69bd834079"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:05:44.802822 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.802802    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bda4e888-9a66-4a7a-bb7c-ef69bd834079" (UID: "bda4e888-9a66-4a7a-bb7c-ef69bd834079"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:44.803275 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.802894    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istiod-ca-cert\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.803275 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.802912    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-workload-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.803275 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.802936    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-data" (OuterVolumeSpecName: "istio-data") pod "bda4e888-9a66-4a7a-bb7c-ef69bd834079" (UID: "bda4e888-9a66-4a7a-bb7c-ef69bd834079"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:44.803275 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.803034    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bda4e888-9a66-4a7a-bb7c-ef69bd834079" (UID: "bda4e888-9a66-4a7a-bb7c-ef69bd834079"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:44.803275 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.803017    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bda4e888-9a66-4a7a-bb7c-ef69bd834079" (UID: "bda4e888-9a66-4a7a-bb7c-ef69bd834079"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:44.804813 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.804781    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-token" (OuterVolumeSpecName: "istio-token") pod "bda4e888-9a66-4a7a-bb7c-ef69bd834079" (UID: "bda4e888-9a66-4a7a-bb7c-ef69bd834079"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:05:44.805169 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.805145    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bda4e888-9a66-4a7a-bb7c-ef69bd834079" (UID: "bda4e888-9a66-4a7a-bb7c-ef69bd834079"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:05:44.805314 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.805288    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bda4e888-9a66-4a7a-bb7c-ef69bd834079" (UID: "bda4e888-9a66-4a7a-bb7c-ef69bd834079"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:05:44.805314 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.805296    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bda4e888-9a66-4a7a-bb7c-ef69bd834079-kube-api-access-k699q" (OuterVolumeSpecName: "kube-api-access-k699q") pod "bda4e888-9a66-4a7a-bb7c-ef69bd834079" (UID: "bda4e888-9a66-4a7a-bb7c-ef69bd834079"). InnerVolumeSpecName "kube-api-access-k699q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:05:44.809286 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.809257    2563 generic.go:356] "Generic (PLEG): container finished" podID="bda4e888-9a66-4a7a-bb7c-ef69bd834079" containerID="119b73b0a805a66b1fbb0f8b9d2ec2eddb2b763613e0c62358350bc3b4848d5c" exitCode=0
Sep 11 14:05:44.809462 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.809307    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"
Sep 11 14:05:44.809462 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.809346    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl" event={"ID":"bda4e888-9a66-4a7a-bb7c-ef69bd834079","Type":"ContainerDied","Data":"119b73b0a805a66b1fbb0f8b9d2ec2eddb2b763613e0c62358350bc3b4848d5c"}
Sep 11 14:05:44.809462 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.809402    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl" event={"ID":"bda4e888-9a66-4a7a-bb7c-ef69bd834079","Type":"ContainerDied","Data":"ea661dc232f5938f1d5c537b8da857bad99511f6d3721ad65df7617f24937d58"}
Sep 11 14:05:44.809462 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.809424    2563 scope.go:117] "RemoveContainer" containerID="119b73b0a805a66b1fbb0f8b9d2ec2eddb2b763613e0c62358350bc3b4848d5c"
Sep 11 14:05:44.822923 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.822897    2563 scope.go:117] "RemoveContainer" containerID="119b73b0a805a66b1fbb0f8b9d2ec2eddb2b763613e0c62358350bc3b4848d5c"
Sep 11 14:05:44.823311 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:05:44.823290    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"119b73b0a805a66b1fbb0f8b9d2ec2eddb2b763613e0c62358350bc3b4848d5c\": container with ID starting with 119b73b0a805a66b1fbb0f8b9d2ec2eddb2b763613e0c62358350bc3b4848d5c not found: ID does not exist" containerID="119b73b0a805a66b1fbb0f8b9d2ec2eddb2b763613e0c62358350bc3b4848d5c"
Sep 11 14:05:44.823377 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.823320    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"119b73b0a805a66b1fbb0f8b9d2ec2eddb2b763613e0c62358350bc3b4848d5c"} err="failed to get container status \"119b73b0a805a66b1fbb0f8b9d2ec2eddb2b763613e0c62358350bc3b4848d5c\": rpc error: code = NotFound desc = could not find container \"119b73b0a805a66b1fbb0f8b9d2ec2eddb2b763613e0c62358350bc3b4848d5c\": container with ID starting with 119b73b0a805a66b1fbb0f8b9d2ec2eddb2b763613e0c62358350bc3b4848d5c not found: ID does not exist"
Sep 11 14:05:44.832867 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.832832    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"]
Sep 11 14:05:44.839934 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.839903    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-4edac4-openshift-default-5f5487cf98-lkqpl"]
Sep 11 14:05:44.904279 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.904241    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-podinfo\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.904279 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.904270    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-credential-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.904279 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.904281    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k699q\" (UniqueName: \"kubernetes.io/projected/bda4e888-9a66-4a7a-bb7c-ef69bd834079-kube-api-access-k699q\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.904510 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.904291    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-token\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.904510 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.904302    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-envoy\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.904510 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.904310    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-workload-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:44.904510 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:44.904317    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bda4e888-9a66-4a7a-bb7c-ef69bd834079-istio-data\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:45.131520 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:45.131486    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bda4e888-9a66-4a7a-bb7c-ef69bd834079" path="/var/lib/kubelet/pods/bda4e888-9a66-4a7a-bb7c-ef69bd834079/volumes"
Sep 11 14:05:49.436589 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.436549    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"]
Sep 11 14:05:49.437136 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.437104    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="338e2621-d789-41c8-9655-c61e206c8b8f" containerName="maas-api"
Sep 11 14:05:49.437136 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.437123    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="338e2621-d789-41c8-9655-c61e206c8b8f" containerName="maas-api"
Sep 11 14:05:49.437136 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.437135    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="159a6029-bfb8-4d10-925a-dc8b2a2b4445" containerName="istio-proxy"
Sep 11 14:05:49.437275 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.437144    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="159a6029-bfb8-4d10-925a-dc8b2a2b4445" containerName="istio-proxy"
Sep 11 14:05:49.437275 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.437156    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="159a6029-bfb8-4d10-925a-dc8b2a2b4445" containerName="istio-proxy"
Sep 11 14:05:49.437275 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.437166    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="159a6029-bfb8-4d10-925a-dc8b2a2b4445" containerName="istio-proxy"
Sep 11 14:05:49.437275 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.437175    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bda4e888-9a66-4a7a-bb7c-ef69bd834079" containerName="istio-proxy"
Sep 11 14:05:49.437275 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.437190    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bda4e888-9a66-4a7a-bb7c-ef69bd834079" containerName="istio-proxy"
Sep 11 14:05:49.437532 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.437278    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="159a6029-bfb8-4d10-925a-dc8b2a2b4445" containerName="istio-proxy"
Sep 11 14:05:49.437532 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.437290    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bda4e888-9a66-4a7a-bb7c-ef69bd834079" containerName="istio-proxy"
Sep 11 14:05:49.437532 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.437300    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="338e2621-d789-41c8-9655-c61e206c8b8f" containerName="maas-api"
Sep 11 14:05:49.437532 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.437313    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f174add-81da-45ba-bd01-a56b94141800" containerName="istio-proxy"
Sep 11 14:05:49.440855 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.440814    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.443208 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.443177    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ffbdd858-openshift-default-dockercfg-tphx6\""
Sep 11 14:05:49.451073 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.451032    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"]
Sep 11 14:05:49.541904 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.541865    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-workload-certs\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.542121 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.541920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-credential-socket\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.542121 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.541946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-envoy\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.542121 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.541978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-data\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.542121 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.542013    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-workload-socket\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.542121 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.542062    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5421fc63-6972-4823-b575-6170f9bcc2e4-istiod-ca-cert\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.542121 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.542100    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpgw6\" (UniqueName: \"kubernetes.io/projected/5421fc63-6972-4823-b575-6170f9bcc2e4-kube-api-access-hpgw6\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.542381 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.542140    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-token\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.542381 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.542198    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-podinfo\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.603842 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.603806    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"]
Sep 11 14:05:49.604088 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.604059    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg" podUID="72e0ba69-636e-4914-b7cf-3fbfb83e311d" containerName="maas-api" containerID="cri-o://0eace4c2cc17e21765d33ef7dd96338fe3e84971db01fddbb11a8fe498b6d4cc" gracePeriod=30
Sep 11 14:05:49.643487 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.643439    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-credential-socket\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.643719 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.643497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-envoy\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.643719 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.643538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-data\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.643719 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.643581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-workload-socket\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.643719 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.643653    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5421fc63-6972-4823-b575-6170f9bcc2e4-istiod-ca-cert\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.643719 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.643684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpgw6\" (UniqueName: \"kubernetes.io/projected/5421fc63-6972-4823-b575-6170f9bcc2e4-kube-api-access-hpgw6\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.643719 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.643710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-token\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.644039 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.643765    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-podinfo\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.644039 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.643806    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-workload-certs\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.644039 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.643935    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-credential-socket\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.644525 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.644477    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-workload-certs\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.644679 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.644582    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5421fc63-6972-4823-b575-6170f9bcc2e4-istiod-ca-cert\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.645159 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.645132    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-data\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.645284 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.645161    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-workload-socket\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.646216 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.646191    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-envoy\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.646829 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.646794    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-podinfo\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.660951 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.660082    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-token\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.661172 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.661135    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpgw6\" (UniqueName: \"kubernetes.io/projected/5421fc63-6972-4823-b575-6170f9bcc2e4-kube-api-access-hpgw6\") pod \"e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") " pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:49.754031 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:49.753942    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:50.029696 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.029667    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"]
Sep 11 14:05:50.032184 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:05:50.032155    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5421fc63_6972_4823_b575_6170f9bcc2e4.slice/crio-385bf8dde44693d09911a478c5383741cd35ea9fed4d81cb5e00a579b5a42c98 WatchSource:0}: Error finding container 385bf8dde44693d09911a478c5383741cd35ea9fed4d81cb5e00a579b5a42c98: Status 404 returned error can't find the container with id 385bf8dde44693d09911a478c5383741cd35ea9fed4d81cb5e00a579b5a42c98
Sep 11 14:05:50.034889 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.034854    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:05:50.035010 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.034926    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:05:50.035010 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.034955    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:05:50.051795 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.051770    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:05:50.148513 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.148472    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72e0ba69-636e-4914-b7cf-3fbfb83e311d-metrics-certs\") pod \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") "
Sep 11 14:05:50.148739 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.148569    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e0ba69-636e-4914-b7cf-3fbfb83e311d-maas-api-tls\") pod \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") "
Sep 11 14:05:50.148739 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.148669    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e0ba69-636e-4914-b7cf-3fbfb83e311d-openshift-service-ca\") pod \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") "
Sep 11 14:05:50.148873 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.148836    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjkjw\" (UniqueName: \"kubernetes.io/projected/72e0ba69-636e-4914-b7cf-3fbfb83e311d-kube-api-access-qjkjw\") pod \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\" (UID: \"72e0ba69-636e-4914-b7cf-3fbfb83e311d\") "
Sep 11 14:05:50.149271 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.149004    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72e0ba69-636e-4914-b7cf-3fbfb83e311d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "72e0ba69-636e-4914-b7cf-3fbfb83e311d" (UID: "72e0ba69-636e-4914-b7cf-3fbfb83e311d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:05:50.149271 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.149168    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e0ba69-636e-4914-b7cf-3fbfb83e311d-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:50.151040 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.151011    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72e0ba69-636e-4914-b7cf-3fbfb83e311d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "72e0ba69-636e-4914-b7cf-3fbfb83e311d" (UID: "72e0ba69-636e-4914-b7cf-3fbfb83e311d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:05:50.151350 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.151321    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72e0ba69-636e-4914-b7cf-3fbfb83e311d-kube-api-access-qjkjw" (OuterVolumeSpecName: "kube-api-access-qjkjw") pod "72e0ba69-636e-4914-b7cf-3fbfb83e311d" (UID: "72e0ba69-636e-4914-b7cf-3fbfb83e311d"). InnerVolumeSpecName "kube-api-access-qjkjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:05:50.152001 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.151974    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72e0ba69-636e-4914-b7cf-3fbfb83e311d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "72e0ba69-636e-4914-b7cf-3fbfb83e311d" (UID: "72e0ba69-636e-4914-b7cf-3fbfb83e311d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:05:50.249926 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.249887    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e0ba69-636e-4914-b7cf-3fbfb83e311d-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:50.249926 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.249921    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjkjw\" (UniqueName: \"kubernetes.io/projected/72e0ba69-636e-4914-b7cf-3fbfb83e311d-kube-api-access-qjkjw\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:50.249926 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.249933    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72e0ba69-636e-4914-b7cf-3fbfb83e311d-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:05:50.845736 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.845696    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj" event={"ID":"5421fc63-6972-4823-b575-6170f9bcc2e4","Type":"ContainerStarted","Data":"3198237717a991c2c9eeabd5c3592331cc97d7cdde045bdc5f3b1e3b1e733096"}
Sep 11 14:05:50.846217 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.845751    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj" event={"ID":"5421fc63-6972-4823-b575-6170f9bcc2e4","Type":"ContainerStarted","Data":"385bf8dde44693d09911a478c5383741cd35ea9fed4d81cb5e00a579b5a42c98"}
Sep 11 14:05:50.847176 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.847150    2563 generic.go:356] "Generic (PLEG): container finished" podID="72e0ba69-636e-4914-b7cf-3fbfb83e311d" containerID="0eace4c2cc17e21765d33ef7dd96338fe3e84971db01fddbb11a8fe498b6d4cc" exitCode=0
Sep 11 14:05:50.847310 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.847192    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"
Sep 11 14:05:50.847310 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.847237    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg" event={"ID":"72e0ba69-636e-4914-b7cf-3fbfb83e311d","Type":"ContainerDied","Data":"0eace4c2cc17e21765d33ef7dd96338fe3e84971db01fddbb11a8fe498b6d4cc"}
Sep 11 14:05:50.847310 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.847295    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg" event={"ID":"72e0ba69-636e-4914-b7cf-3fbfb83e311d","Type":"ContainerDied","Data":"d8600c00d50330aeb5e6b1c4e6c715a0cae54ef147732e849028c2ac3e9131f9"}
Sep 11 14:05:50.847492 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.847333    2563 scope.go:117] "RemoveContainer" containerID="0eace4c2cc17e21765d33ef7dd96338fe3e84971db01fddbb11a8fe498b6d4cc"
Sep 11 14:05:50.857801 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.857780    2563 scope.go:117] "RemoveContainer" containerID="0eace4c2cc17e21765d33ef7dd96338fe3e84971db01fddbb11a8fe498b6d4cc"
Sep 11 14:05:50.858137 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:05:50.858112    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eace4c2cc17e21765d33ef7dd96338fe3e84971db01fddbb11a8fe498b6d4cc\": container with ID starting with 0eace4c2cc17e21765d33ef7dd96338fe3e84971db01fddbb11a8fe498b6d4cc not found: ID does not exist" containerID="0eace4c2cc17e21765d33ef7dd96338fe3e84971db01fddbb11a8fe498b6d4cc"
Sep 11 14:05:50.858231 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.858148    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eace4c2cc17e21765d33ef7dd96338fe3e84971db01fddbb11a8fe498b6d4cc"} err="failed to get container status \"0eace4c2cc17e21765d33ef7dd96338fe3e84971db01fddbb11a8fe498b6d4cc\": rpc error: code = NotFound desc = could not find container \"0eace4c2cc17e21765d33ef7dd96338fe3e84971db01fddbb11a8fe498b6d4cc\": container with ID starting with 0eace4c2cc17e21765d33ef7dd96338fe3e84971db01fddbb11a8fe498b6d4cc not found: ID does not exist"
Sep 11 14:05:50.867969 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.867918    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj" podStartSLOduration=1.8679019650000002 podStartE2EDuration="1.867901965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:05:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:05:50.866155203 +0000 UTC m=+1032.263497941" watchObservedRunningTime="2026-09-11 14:05:50.867901965 +0000 UTC m=+1032.265244713"
Sep 11 14:05:50.881576 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.881539    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"]
Sep 11 14:05:50.884728 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:50.884698    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-da9534-c7c9f95d7-qh4hg"]
Sep 11 14:05:51.131868 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:51.131833    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72e0ba69-636e-4914-b7cf-3fbfb83e311d" path="/var/lib/kubelet/pods/72e0ba69-636e-4914-b7cf-3fbfb83e311d/volumes"
Sep 11 14:05:51.754502 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:51.754455    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:51.759758 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:51.759726    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:51.853589 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:51.853546    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:51.854521 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:51.854500    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:05:54.010684 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.010650    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"]
Sep 11 14:05:54.011051 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.011016    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72e0ba69-636e-4914-b7cf-3fbfb83e311d" containerName="maas-api"
Sep 11 14:05:54.011051 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.011029    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="72e0ba69-636e-4914-b7cf-3fbfb83e311d" containerName="maas-api"
Sep 11 14:05:54.011127 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.011089    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="72e0ba69-636e-4914-b7cf-3fbfb83e311d" containerName="maas-api"
Sep 11 14:05:54.011127 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.011102    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="159a6029-bfb8-4d10-925a-dc8b2a2b4445" containerName="istio-proxy"
Sep 11 14:05:54.014645 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.014595    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:54.016777 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.016747    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ffbdd858-serving-cert\""
Sep 11 14:05:54.027333 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.027302    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"]
Sep 11 14:05:54.084416 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.084372    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da7c1374-eb14-46c4-b50b-1b30dfd7f001-metrics-certs\") pod \"maas-api-e2e-mt-ffbdd858-959c6b457-sv28g\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:54.084416 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.084417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da7c1374-eb14-46c4-b50b-1b30dfd7f001-maas-api-tls\") pod \"maas-api-e2e-mt-ffbdd858-959c6b457-sv28g\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:54.084764 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.084479    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da7c1374-eb14-46c4-b50b-1b30dfd7f001-openshift-service-ca\") pod \"maas-api-e2e-mt-ffbdd858-959c6b457-sv28g\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:54.084764 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.084500    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r47b6\" (UniqueName: \"kubernetes.io/projected/da7c1374-eb14-46c4-b50b-1b30dfd7f001-kube-api-access-r47b6\") pod \"maas-api-e2e-mt-ffbdd858-959c6b457-sv28g\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:54.185828 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.185785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da7c1374-eb14-46c4-b50b-1b30dfd7f001-openshift-service-ca\") pod \"maas-api-e2e-mt-ffbdd858-959c6b457-sv28g\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:54.185828 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.185829    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r47b6\" (UniqueName: \"kubernetes.io/projected/da7c1374-eb14-46c4-b50b-1b30dfd7f001-kube-api-access-r47b6\") pod \"maas-api-e2e-mt-ffbdd858-959c6b457-sv28g\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:54.186093 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.185917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da7c1374-eb14-46c4-b50b-1b30dfd7f001-metrics-certs\") pod \"maas-api-e2e-mt-ffbdd858-959c6b457-sv28g\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:54.186093 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.185968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da7c1374-eb14-46c4-b50b-1b30dfd7f001-maas-api-tls\") pod \"maas-api-e2e-mt-ffbdd858-959c6b457-sv28g\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:54.186210 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:05:54.186115    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-serving-cert: secret "maas-api-e2e-mt-ffbdd858-serving-cert" not found
Sep 11 14:05:54.186210 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:05:54.186180    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da7c1374-eb14-46c4-b50b-1b30dfd7f001-maas-api-tls podName:da7c1374-eb14-46c4-b50b-1b30dfd7f001 nodeName:}" failed. No retries permitted until 2026-09-11 14:05:54.686161211 +0000 UTC m=+1036.083503925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/da7c1374-eb14-46c4-b50b-1b30dfd7f001-maas-api-tls") pod "maas-api-e2e-mt-ffbdd858-959c6b457-sv28g" (UID: "da7c1374-eb14-46c4-b50b-1b30dfd7f001") : secret "maas-api-e2e-mt-ffbdd858-serving-cert" not found
Sep 11 14:05:54.186489 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.186467    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da7c1374-eb14-46c4-b50b-1b30dfd7f001-openshift-service-ca\") pod \"maas-api-e2e-mt-ffbdd858-959c6b457-sv28g\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:54.188695 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.188667    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da7c1374-eb14-46c4-b50b-1b30dfd7f001-metrics-certs\") pod \"maas-api-e2e-mt-ffbdd858-959c6b457-sv28g\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:54.197164 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.197132    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r47b6\" (UniqueName: \"kubernetes.io/projected/da7c1374-eb14-46c4-b50b-1b30dfd7f001-kube-api-access-r47b6\") pod \"maas-api-e2e-mt-ffbdd858-959c6b457-sv28g\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:54.690465 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.690423    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da7c1374-eb14-46c4-b50b-1b30dfd7f001-maas-api-tls\") pod \"maas-api-e2e-mt-ffbdd858-959c6b457-sv28g\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:54.693147 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.693117    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da7c1374-eb14-46c4-b50b-1b30dfd7f001-maas-api-tls\") pod \"maas-api-e2e-mt-ffbdd858-959c6b457-sv28g\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:54.927325 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:54.927281    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:55.200107 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:55.200078    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"]
Sep 11 14:05:55.202778 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:05:55.202746    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda7c1374_eb14_46c4_b50b_1b30dfd7f001.slice/crio-08b4f92263764f036f7c18696636bb85b9c9f0a70a9249bb88783de1edd3acce WatchSource:0}: Error finding container 08b4f92263764f036f7c18696636bb85b9c9f0a70a9249bb88783de1edd3acce: Status 404 returned error can't find the container with id 08b4f92263764f036f7c18696636bb85b9c9f0a70a9249bb88783de1edd3acce
Sep 11 14:05:55.204270 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:55.204248    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:05:55.872966 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:55.872925    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g" event={"ID":"da7c1374-eb14-46c4-b50b-1b30dfd7f001","Type":"ContainerStarted","Data":"bbbbd19b82eb6cf782a2bd491b01675c5667ae90611d9ffc941ad064a183dc21"}
Sep 11 14:05:55.873223 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:55.872993    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g" event={"ID":"da7c1374-eb14-46c4-b50b-1b30dfd7f001","Type":"ContainerStarted","Data":"08b4f92263764f036f7c18696636bb85b9c9f0a70a9249bb88783de1edd3acce"}
Sep 11 14:05:55.873223 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:55.873082    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:05:55.889617 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:05:55.889543    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g" podStartSLOduration=2.395986777 podStartE2EDuration="2.889518816s" totalImagesPullingTime="493.532039ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:05:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:05:55.889029929 +0000 UTC m=+1037.286372667" watchObservedRunningTime="2026-09-11 14:05:55.889518816 +0000 UTC m=+1037.286861564"
Sep 11 14:06:01.892185 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:01.892141    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:06:02.266914 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.266816    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"]
Sep 11 14:06:02.269932 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.269906    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:02.275944 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.275912    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-932c97-serving-cert\""
Sep 11 14:06:02.285858 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.285821    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"]
Sep 11 14:06:02.363227 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.363181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njswn\" (UniqueName: \"kubernetes.io/projected/97297e22-0df7-4a1d-bc6d-c3255e415e6a-kube-api-access-njswn\") pod \"maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:02.363445 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.363327    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97297e22-0df7-4a1d-bc6d-c3255e415e6a-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:02.363445 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.363363    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97297e22-0df7-4a1d-bc6d-c3255e415e6a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:02.363445 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.363395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97297e22-0df7-4a1d-bc6d-c3255e415e6a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:02.464206 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.464160    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97297e22-0df7-4a1d-bc6d-c3255e415e6a-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:02.464206 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.464208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97297e22-0df7-4a1d-bc6d-c3255e415e6a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:02.464484 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.464227    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97297e22-0df7-4a1d-bc6d-c3255e415e6a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:02.464484 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.464261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njswn\" (UniqueName: \"kubernetes.io/projected/97297e22-0df7-4a1d-bc6d-c3255e415e6a-kube-api-access-njswn\") pod \"maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:02.464595 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:06:02.464503    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-serving-cert: secret "maas-api-e2e-shared-a-w3-932c97-serving-cert" not found
Sep 11 14:06:02.464679 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:06:02.464602    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97297e22-0df7-4a1d-bc6d-c3255e415e6a-maas-api-tls podName:97297e22-0df7-4a1d-bc6d-c3255e415e6a nodeName:}" failed. No retries permitted until 2026-09-11 14:06:02.964573485 +0000 UTC m=+1044.361916206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/97297e22-0df7-4a1d-bc6d-c3255e415e6a-maas-api-tls") pod "maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc" (UID: "97297e22-0df7-4a1d-bc6d-c3255e415e6a") : secret "maas-api-e2e-shared-a-w3-932c97-serving-cert" not found
Sep 11 14:06:02.465022 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.464997    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97297e22-0df7-4a1d-bc6d-c3255e415e6a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:02.467102 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.467073    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97297e22-0df7-4a1d-bc6d-c3255e415e6a-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:02.474707 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.474671    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njswn\" (UniqueName: \"kubernetes.io/projected/97297e22-0df7-4a1d-bc6d-c3255e415e6a-kube-api-access-njswn\") pod \"maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:02.970206 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.970158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97297e22-0df7-4a1d-bc6d-c3255e415e6a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:02.979653 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:02.977245    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97297e22-0df7-4a1d-bc6d-c3255e415e6a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:03.212733 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:03.212688    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:03.489462 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:03.489355    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"]
Sep 11 14:06:03.918689 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:03.918648    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc" event={"ID":"97297e22-0df7-4a1d-bc6d-c3255e415e6a","Type":"ContainerStarted","Data":"a8cb6704fdb6d1ba0d76fcc0b75358e584793fa60d3e2d9194675417dcfb5d8c"}
Sep 11 14:06:04.925682 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:04.925611    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc" event={"ID":"97297e22-0df7-4a1d-bc6d-c3255e415e6a","Type":"ContainerStarted","Data":"d9997cddbb1b1f52c6060fbdf28612d2809b23974bc0c3d325fd9f961df773c0"}
Sep 11 14:06:04.926173 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:04.925798    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:04.944115 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:04.944058    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc" podStartSLOduration=2.513245075 podStartE2EDuration="2.944040103s" totalImagesPullingTime="430.795028ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:06:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:04.942691405 +0000 UTC m=+1046.340034143" watchObservedRunningTime="2026-09-11 14:06:04.944040103 +0000 UTC m=+1046.341382839"
Sep 11 14:06:07.270114 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:07.270077    2563 patch_prober.go:28] interesting pod/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": read tcp 10.133.0.2:42042->10.133.0.42:15021: read: connection reset by peer" start-of-body=
Sep 11 14:06:07.270525 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:07.270169    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj" podUID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": read tcp 10.133.0.2:42042->10.133.0.42:15021: read: connection reset by peer"
Sep 11 14:06:07.942528 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:07.942494    2563 generic.go:356] "Generic (PLEG): container finished" podID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerID="3198237717a991c2c9eeabd5c3592331cc97d7cdde045bdc5f3b1e3b1e733096" exitCode=0
Sep 11 14:06:07.942739 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:07.942569    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj" event={"ID":"5421fc63-6972-4823-b575-6170f9bcc2e4","Type":"ContainerDied","Data":"3198237717a991c2c9eeabd5c3592331cc97d7cdde045bdc5f3b1e3b1e733096"}
Sep 11 14:06:07.943011 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:07.942994    2563 scope.go:117] "RemoveContainer" containerID="3198237717a991c2c9eeabd5c3592331cc97d7cdde045bdc5f3b1e3b1e733096"
Sep 11 14:06:07.944081 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:07.944056    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:06:07.944143 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:07.944126    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:06:07.944183 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:07.944157    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:06:08.949457 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:08.949416    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj" event={"ID":"5421fc63-6972-4823-b575-6170f9bcc2e4","Type":"ContainerStarted","Data":"da1bafefb2cc271077dc12fce0d19b2b22735b70c6c6384eb79af921bb6c9d7d"}
Sep 11 14:06:09.754350 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:09.754306    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:06:09.754612 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:09.754590    2563 patch_prober.go:28] interesting pod/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 11 14:06:09.754755 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:09.754684    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj" podUID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 11 14:06:10.754956 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:10.754914    2563 patch_prober.go:28] interesting pod/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 11 14:06:10.755363 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:10.754982    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj" podUID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 11 14:06:10.937547 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:10.937517    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:06:11.754806 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:11.754767    2563 patch_prober.go:28] interesting pod/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 11 14:06:11.755001 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:11.754825    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj" podUID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 11 14:06:12.754773 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:12.754734    2563 patch_prober.go:28] interesting pod/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 11 14:06:12.754979 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:12.754804    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj" podUID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 11 14:06:13.754883 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:13.754846    2563 patch_prober.go:28] interesting pod/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 11 14:06:13.755261 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:13.754905    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj" podUID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 11 14:06:14.758564 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:14.758528    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:06:14.758996 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:14.758925    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:06:14.759645 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:14.759602    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:06:30.716893 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:30.716799    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"]
Sep 11 14:06:30.720912 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:30.720882    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:30.723119 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:30.723097    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-367517-serving-cert\""
Sep 11 14:06:30.733070 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:30.733026    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"]
Sep 11 14:06:30.849377 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:30.849330    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:30.849586 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:30.849387    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be8da740-4454-471b-9b33-77afd2fc44e3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:30.849586 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:30.849539    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfqh2\" (UniqueName: \"kubernetes.io/projected/be8da740-4454-471b-9b33-77afd2fc44e3-kube-api-access-wfqh2\") pod \"maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:30.849732 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:30.849588    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:30.950879 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:30.950839    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfqh2\" (UniqueName: \"kubernetes.io/projected/be8da740-4454-471b-9b33-77afd2fc44e3-kube-api-access-wfqh2\") pod \"maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:30.951065 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:30.950893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:30.951065 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:30.950941    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:30.951065 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:30.950968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be8da740-4454-471b-9b33-77afd2fc44e3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:30.951194 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:06:30.951092    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-serving-cert: secret "maas-api-e2e-shared-b-w3-367517-serving-cert" not found
Sep 11 14:06:30.951194 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:06:30.951175    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-maas-api-tls podName:be8da740-4454-471b-9b33-77afd2fc44e3 nodeName:}" failed. No retries permitted until 2026-09-11 14:06:31.451155861 +0000 UTC m=+1072.848498577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-maas-api-tls") pod "maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq" (UID: "be8da740-4454-471b-9b33-77afd2fc44e3") : secret "maas-api-e2e-shared-b-w3-367517-serving-cert" not found
Sep 11 14:06:30.951807 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:30.951732    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be8da740-4454-471b-9b33-77afd2fc44e3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:30.953777 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:30.953751    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:30.962340 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:30.962306    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfqh2\" (UniqueName: \"kubernetes.io/projected/be8da740-4454-471b-9b33-77afd2fc44e3-kube-api-access-wfqh2\") pod \"maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:31.456757 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:31.456698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:31.456992 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:06:31.456856    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-serving-cert: secret "maas-api-e2e-shared-b-w3-367517-serving-cert" not found
Sep 11 14:06:31.456992 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:06:31.456928    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-maas-api-tls podName:be8da740-4454-471b-9b33-77afd2fc44e3 nodeName:}" failed. No retries permitted until 2026-09-11 14:06:32.456909538 +0000 UTC m=+1073.854252275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-maas-api-tls") pod "maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq" (UID: "be8da740-4454-471b-9b33-77afd2fc44e3") : secret "maas-api-e2e-shared-b-w3-367517-serving-cert" not found
Sep 11 14:06:32.467425 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:32.467378    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:32.470241 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:32.470217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:32.535427 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:32.535380    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:32.808188 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:32.808162    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"]
Sep 11 14:06:32.810981 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:06:32.810946    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe8da740_4454_471b_9b33_77afd2fc44e3.slice/crio-78e04c6ba783f19a7522368c9ce8fcd2444a373b24fe5f11c095da1240f8155b WatchSource:0}: Error finding container 78e04c6ba783f19a7522368c9ce8fcd2444a373b24fe5f11c095da1240f8155b: Status 404 returned error can't find the container with id 78e04c6ba783f19a7522368c9ce8fcd2444a373b24fe5f11c095da1240f8155b
Sep 11 14:06:33.083078 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:33.082976    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq" event={"ID":"be8da740-4454-471b-9b33-77afd2fc44e3","Type":"ContainerStarted","Data":"78e04c6ba783f19a7522368c9ce8fcd2444a373b24fe5f11c095da1240f8155b"}
Sep 11 14:06:34.089308 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:34.089270    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq" event={"ID":"be8da740-4454-471b-9b33-77afd2fc44e3","Type":"ContainerStarted","Data":"00201aad4047644a30d74cd734c829868c6529b2ca802b52d8e1d41514ab73dc"}
Sep 11 14:06:34.089761 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:34.089338    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:34.111360 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:34.111297    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq" podStartSLOduration=3.676161553 podStartE2EDuration="4.111276042s" totalImagesPullingTime="435.114489ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:06:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:34.1093545 +0000 UTC m=+1075.506697238" watchObservedRunningTime="2026-09-11 14:06:34.111276042 +0000 UTC m=+1075.508618819"
Sep 11 14:06:35.993061 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:35.993019    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"]
Sep 11 14:06:35.993549 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:35.993345    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g" podUID="da7c1374-eb14-46c4-b50b-1b30dfd7f001" containerName="maas-api" containerID="cri-o://bbbbd19b82eb6cf782a2bd491b01675c5667ae90611d9ffc941ad064a183dc21" gracePeriod=30
Sep 11 14:06:36.257531 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:36.257459    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:06:36.403217 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:36.403179    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da7c1374-eb14-46c4-b50b-1b30dfd7f001-maas-api-tls\") pod \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") "
Sep 11 14:06:36.403404 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:36.403235    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da7c1374-eb14-46c4-b50b-1b30dfd7f001-metrics-certs\") pod \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") "
Sep 11 14:06:36.403404 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:36.403287    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da7c1374-eb14-46c4-b50b-1b30dfd7f001-openshift-service-ca\") pod \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") "
Sep 11 14:06:36.403404 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:36.403322    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r47b6\" (UniqueName: \"kubernetes.io/projected/da7c1374-eb14-46c4-b50b-1b30dfd7f001-kube-api-access-r47b6\") pod \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\" (UID: \"da7c1374-eb14-46c4-b50b-1b30dfd7f001\") "
Sep 11 14:06:36.403747 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:36.403716    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da7c1374-eb14-46c4-b50b-1b30dfd7f001-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "da7c1374-eb14-46c4-b50b-1b30dfd7f001" (UID: "da7c1374-eb14-46c4-b50b-1b30dfd7f001"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:06:36.405970 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:36.405933    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da7c1374-eb14-46c4-b50b-1b30dfd7f001-kube-api-access-r47b6" (OuterVolumeSpecName: "kube-api-access-r47b6") pod "da7c1374-eb14-46c4-b50b-1b30dfd7f001" (UID: "da7c1374-eb14-46c4-b50b-1b30dfd7f001"). InnerVolumeSpecName "kube-api-access-r47b6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:06:36.406199 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:36.406172    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da7c1374-eb14-46c4-b50b-1b30dfd7f001-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "da7c1374-eb14-46c4-b50b-1b30dfd7f001" (UID: "da7c1374-eb14-46c4-b50b-1b30dfd7f001"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:06:36.406199 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:36.406183    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da7c1374-eb14-46c4-b50b-1b30dfd7f001-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "da7c1374-eb14-46c4-b50b-1b30dfd7f001" (UID: "da7c1374-eb14-46c4-b50b-1b30dfd7f001"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:06:36.504948 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:36.504901    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da7c1374-eb14-46c4-b50b-1b30dfd7f001-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:06:36.504948 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:36.504939    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da7c1374-eb14-46c4-b50b-1b30dfd7f001-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:06:36.504948 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:36.504949    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da7c1374-eb14-46c4-b50b-1b30dfd7f001-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:06:36.504948 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:36.504959    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r47b6\" (UniqueName: \"kubernetes.io/projected/da7c1374-eb14-46c4-b50b-1b30dfd7f001-kube-api-access-r47b6\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:06:37.105434 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:37.105396    2563 generic.go:356] "Generic (PLEG): container finished" podID="da7c1374-eb14-46c4-b50b-1b30dfd7f001" containerID="bbbbd19b82eb6cf782a2bd491b01675c5667ae90611d9ffc941ad064a183dc21" exitCode=0
Sep 11 14:06:37.105974 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:37.105451    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"
Sep 11 14:06:37.105974 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:37.105478    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g" event={"ID":"da7c1374-eb14-46c4-b50b-1b30dfd7f001","Type":"ContainerDied","Data":"bbbbd19b82eb6cf782a2bd491b01675c5667ae90611d9ffc941ad064a183dc21"}
Sep 11 14:06:37.105974 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:37.105531    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g" event={"ID":"da7c1374-eb14-46c4-b50b-1b30dfd7f001","Type":"ContainerDied","Data":"08b4f92263764f036f7c18696636bb85b9c9f0a70a9249bb88783de1edd3acce"}
Sep 11 14:06:37.105974 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:37.105554    2563 scope.go:117] "RemoveContainer" containerID="bbbbd19b82eb6cf782a2bd491b01675c5667ae90611d9ffc941ad064a183dc21"
Sep 11 14:06:37.114773 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:37.114754    2563 scope.go:117] "RemoveContainer" containerID="bbbbd19b82eb6cf782a2bd491b01675c5667ae90611d9ffc941ad064a183dc21"
Sep 11 14:06:37.115093 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:06:37.115074    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbbbd19b82eb6cf782a2bd491b01675c5667ae90611d9ffc941ad064a183dc21\": container with ID starting with bbbbd19b82eb6cf782a2bd491b01675c5667ae90611d9ffc941ad064a183dc21 not found: ID does not exist" containerID="bbbbd19b82eb6cf782a2bd491b01675c5667ae90611d9ffc941ad064a183dc21"
Sep 11 14:06:37.115144 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:37.115103    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbbbd19b82eb6cf782a2bd491b01675c5667ae90611d9ffc941ad064a183dc21"} err="failed to get container status \"bbbbd19b82eb6cf782a2bd491b01675c5667ae90611d9ffc941ad064a183dc21\": rpc error: code = NotFound desc = could not find container \"bbbbd19b82eb6cf782a2bd491b01675c5667ae90611d9ffc941ad064a183dc21\": container with ID starting with bbbbd19b82eb6cf782a2bd491b01675c5667ae90611d9ffc941ad064a183dc21 not found: ID does not exist"
Sep 11 14:06:37.131519 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:37.131490    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"]
Sep 11 14:06:37.131711 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:37.131529    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffbdd858-959c6b457-sv28g"]
Sep 11 14:06:39.132016 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:39.131987    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da7c1374-eb14-46c4-b50b-1b30dfd7f001" path="/var/lib/kubelet/pods/da7c1374-eb14-46c4-b50b-1b30dfd7f001/volumes"
Sep 11 14:06:40.100409 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:40.100378    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:06:42.616274 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:42.616236    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"]
Sep 11 14:06:42.616762 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:42.616504    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj" podUID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerName="istio-proxy" containerID="cri-o://da1bafefb2cc271077dc12fce0d19b2b22735b70c6c6384eb79af921bb6c9d7d" gracePeriod=30
Sep 11 14:06:44.761697 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:44.761662    2563 patch_prober.go:28] interesting pod/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 14:06:44.762079 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:44.761719    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj" podUID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 14:06:47.883460 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:47.883373    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:06:48.008685 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.008611    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-token\") pod \"5421fc63-6972-4823-b575-6170f9bcc2e4\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") "
Sep 11 14:06:48.008685 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.008682    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-workload-certs\") pod \"5421fc63-6972-4823-b575-6170f9bcc2e4\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") "
Sep 11 14:06:48.008903 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.008717    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-podinfo\") pod \"5421fc63-6972-4823-b575-6170f9bcc2e4\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") "
Sep 11 14:06:48.008903 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.008737    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-data\") pod \"5421fc63-6972-4823-b575-6170f9bcc2e4\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") "
Sep 11 14:06:48.008903 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.008752    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5421fc63-6972-4823-b575-6170f9bcc2e4-istiod-ca-cert\") pod \"5421fc63-6972-4823-b575-6170f9bcc2e4\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") "
Sep 11 14:06:48.008903 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.008783    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-credential-socket\") pod \"5421fc63-6972-4823-b575-6170f9bcc2e4\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") "
Sep 11 14:06:48.008903 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.008810    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-workload-socket\") pod \"5421fc63-6972-4823-b575-6170f9bcc2e4\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") "
Sep 11 14:06:48.008903 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.008867    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpgw6\" (UniqueName: \"kubernetes.io/projected/5421fc63-6972-4823-b575-6170f9bcc2e4-kube-api-access-hpgw6\") pod \"5421fc63-6972-4823-b575-6170f9bcc2e4\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") "
Sep 11 14:06:48.009244 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.008907    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-envoy\") pod \"5421fc63-6972-4823-b575-6170f9bcc2e4\" (UID: \"5421fc63-6972-4823-b575-6170f9bcc2e4\") "
Sep 11 14:06:48.009244 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.009083    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-data" (OuterVolumeSpecName: "istio-data") pod "5421fc63-6972-4823-b575-6170f9bcc2e4" (UID: "5421fc63-6972-4823-b575-6170f9bcc2e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:06:48.009244 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.009095    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5421fc63-6972-4823-b575-6170f9bcc2e4" (UID: "5421fc63-6972-4823-b575-6170f9bcc2e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:06:48.009244 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.009111    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5421fc63-6972-4823-b575-6170f9bcc2e4" (UID: "5421fc63-6972-4823-b575-6170f9bcc2e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:06:48.009244 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.009190    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5421fc63-6972-4823-b575-6170f9bcc2e4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5421fc63-6972-4823-b575-6170f9bcc2e4" (UID: "5421fc63-6972-4823-b575-6170f9bcc2e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:06:48.009513 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.009328    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5421fc63-6972-4823-b575-6170f9bcc2e4" (UID: "5421fc63-6972-4823-b575-6170f9bcc2e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:06:48.009513 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.009354    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-workload-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:06:48.009513 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.009373    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-data\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:06:48.009513 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.009386    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5421fc63-6972-4823-b575-6170f9bcc2e4-istiod-ca-cert\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:06:48.009513 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.009398    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-credential-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:06:48.011067 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.011042    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5421fc63-6972-4823-b575-6170f9bcc2e4" (UID: "5421fc63-6972-4823-b575-6170f9bcc2e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:06:48.011251 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.011229    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5421fc63-6972-4823-b575-6170f9bcc2e4-kube-api-access-hpgw6" (OuterVolumeSpecName: "kube-api-access-hpgw6") pod "5421fc63-6972-4823-b575-6170f9bcc2e4" (UID: "5421fc63-6972-4823-b575-6170f9bcc2e4"). InnerVolumeSpecName "kube-api-access-hpgw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:06:48.011539 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.011519    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "5421fc63-6972-4823-b575-6170f9bcc2e4" (UID: "5421fc63-6972-4823-b575-6170f9bcc2e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:06:48.011600 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.011522    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5421fc63-6972-4823-b575-6170f9bcc2e4" (UID: "5421fc63-6972-4823-b575-6170f9bcc2e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:06:48.110232 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.110185    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpgw6\" (UniqueName: \"kubernetes.io/projected/5421fc63-6972-4823-b575-6170f9bcc2e4-kube-api-access-hpgw6\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:06:48.110232 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.110223    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-envoy\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:06:48.110232 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.110237    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-token\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:06:48.110488 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.110249    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5421fc63-6972-4823-b575-6170f9bcc2e4-istio-podinfo\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:06:48.110488 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.110261    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5421fc63-6972-4823-b575-6170f9bcc2e4-workload-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:06:48.160131 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.160044    2563 generic.go:356] "Generic (PLEG): container finished" podID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerID="da1bafefb2cc271077dc12fce0d19b2b22735b70c6c6384eb79af921bb6c9d7d" exitCode=0
Sep 11 14:06:48.160131 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.160097    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"
Sep 11 14:06:48.160318 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.160124    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj" event={"ID":"5421fc63-6972-4823-b575-6170f9bcc2e4","Type":"ContainerDied","Data":"da1bafefb2cc271077dc12fce0d19b2b22735b70c6c6384eb79af921bb6c9d7d"}
Sep 11 14:06:48.160318 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.160188    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj" event={"ID":"5421fc63-6972-4823-b575-6170f9bcc2e4","Type":"ContainerDied","Data":"385bf8dde44693d09911a478c5383741cd35ea9fed4d81cb5e00a579b5a42c98"}
Sep 11 14:06:48.160318 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.160218    2563 scope.go:117] "RemoveContainer" containerID="da1bafefb2cc271077dc12fce0d19b2b22735b70c6c6384eb79af921bb6c9d7d"
Sep 11 14:06:48.170556 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.170536    2563 scope.go:117] "RemoveContainer" containerID="3198237717a991c2c9eeabd5c3592331cc97d7cdde045bdc5f3b1e3b1e733096"
Sep 11 14:06:48.180500 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.180473    2563 scope.go:117] "RemoveContainer" containerID="da1bafefb2cc271077dc12fce0d19b2b22735b70c6c6384eb79af921bb6c9d7d"
Sep 11 14:06:48.180929 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:06:48.180904    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da1bafefb2cc271077dc12fce0d19b2b22735b70c6c6384eb79af921bb6c9d7d\": container with ID starting with da1bafefb2cc271077dc12fce0d19b2b22735b70c6c6384eb79af921bb6c9d7d not found: ID does not exist" containerID="da1bafefb2cc271077dc12fce0d19b2b22735b70c6c6384eb79af921bb6c9d7d"
Sep 11 14:06:48.180997 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.180946    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da1bafefb2cc271077dc12fce0d19b2b22735b70c6c6384eb79af921bb6c9d7d"} err="failed to get container status \"da1bafefb2cc271077dc12fce0d19b2b22735b70c6c6384eb79af921bb6c9d7d\": rpc error: code = NotFound desc = could not find container \"da1bafefb2cc271077dc12fce0d19b2b22735b70c6c6384eb79af921bb6c9d7d\": container with ID starting with da1bafefb2cc271077dc12fce0d19b2b22735b70c6c6384eb79af921bb6c9d7d not found: ID does not exist"
Sep 11 14:06:48.180997 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.180973    2563 scope.go:117] "RemoveContainer" containerID="3198237717a991c2c9eeabd5c3592331cc97d7cdde045bdc5f3b1e3b1e733096"
Sep 11 14:06:48.181350 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:06:48.181324    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3198237717a991c2c9eeabd5c3592331cc97d7cdde045bdc5f3b1e3b1e733096\": container with ID starting with 3198237717a991c2c9eeabd5c3592331cc97d7cdde045bdc5f3b1e3b1e733096 not found: ID does not exist" containerID="3198237717a991c2c9eeabd5c3592331cc97d7cdde045bdc5f3b1e3b1e733096"
Sep 11 14:06:48.181460 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.181351    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3198237717a991c2c9eeabd5c3592331cc97d7cdde045bdc5f3b1e3b1e733096"} err="failed to get container status \"3198237717a991c2c9eeabd5c3592331cc97d7cdde045bdc5f3b1e3b1e733096\": rpc error: code = NotFound desc = could not find container \"3198237717a991c2c9eeabd5c3592331cc97d7cdde045bdc5f3b1e3b1e733096\": container with ID starting with 3198237717a991c2c9eeabd5c3592331cc97d7cdde045bdc5f3b1e3b1e733096 not found: ID does not exist"
Sep 11 14:06:48.182419 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.182398    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"]
Sep 11 14:06:48.186726 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:48.186697    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ffbdd858-openshift-default-5bfc98b9b-w8qmj"]
Sep 11 14:06:49.131014 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:49.130980    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5421fc63-6972-4823-b575-6170f9bcc2e4" path="/var/lib/kubelet/pods/5421fc63-6972-4823-b575-6170f9bcc2e4/volumes"
Sep 11 14:06:56.674088 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.674043    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"]
Sep 11 14:06:56.674593 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.674576    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da7c1374-eb14-46c4-b50b-1b30dfd7f001" containerName="maas-api"
Sep 11 14:06:56.674682 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.674596    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="da7c1374-eb14-46c4-b50b-1b30dfd7f001" containerName="maas-api"
Sep 11 14:06:56.674682 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.674613    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerName="istio-proxy"
Sep 11 14:06:56.674682 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.674621    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerName="istio-proxy"
Sep 11 14:06:56.674682 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.674660    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerName="istio-proxy"
Sep 11 14:06:56.674682 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.674669    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerName="istio-proxy"
Sep 11 14:06:56.674939 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.674750    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerName="istio-proxy"
Sep 11 14:06:56.674939 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.674759    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5421fc63-6972-4823-b575-6170f9bcc2e4" containerName="istio-proxy"
Sep 11 14:06:56.674939 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.674770    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="da7c1374-eb14-46c4-b50b-1b30dfd7f001" containerName="maas-api"
Sep 11 14:06:56.678318 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.678291    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.680871 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.680842    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-6b9470-openshift-default-dockercfg-dmwmg\""
Sep 11 14:06:56.683841 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.683815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-workload-socket\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.683979 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.683848    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-envoy\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.683979 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.683882    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7l9p\" (UniqueName: \"kubernetes.io/projected/407aec10-e5ca-4004-84ca-7deecac5ef3b-kube-api-access-b7l9p\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.683979 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.683931    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-data\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.683979 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.683958    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/407aec10-e5ca-4004-84ca-7deecac5ef3b-istiod-ca-cert\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.684188 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.684000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-workload-certs\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.684188 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.684016    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-podinfo\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.684188 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.684045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-credential-socket\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.684188 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.684080    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-token\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.689513 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.689437    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"]
Sep 11 14:06:56.785162 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.785122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-data\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.785162 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.785169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/407aec10-e5ca-4004-84ca-7deecac5ef3b-istiod-ca-cert\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.785434 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.785221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-workload-certs\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.785434 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.785246    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-podinfo\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.785434 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.785278    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-credential-socket\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.785434 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.785316    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-token\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.785434 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.785354    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-workload-socket\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.785745 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.785671    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-workload-certs\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.785804 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.785741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-envoy\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.785804 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.785791    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7l9p\" (UniqueName: \"kubernetes.io/projected/407aec10-e5ca-4004-84ca-7deecac5ef3b-kube-api-access-b7l9p\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.786105 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.786053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/407aec10-e5ca-4004-84ca-7deecac5ef3b-istiod-ca-cert\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.786239 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.786186    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-workload-socket\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.786348 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.786316    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-data\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.786469 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.786445    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-credential-socket\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.788460 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.788433    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-podinfo\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.788582 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.788565    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-envoy\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.792907 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.792881    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7l9p\" (UniqueName: \"kubernetes.io/projected/407aec10-e5ca-4004-84ca-7deecac5ef3b-kube-api-access-b7l9p\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.793130 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.793112    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-token\") pod \"e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") " pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:56.994704 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:56.994570    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:57.270361 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:57.270329    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"]
Sep 11 14:06:57.273139 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:06:57.273092    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod407aec10_e5ca_4004_84ca_7deecac5ef3b.slice/crio-e9361580d826465fb864e6381de4525e2f8f3feef63c12541140c6c48dc2c6aa WatchSource:0}: Error finding container e9361580d826465fb864e6381de4525e2f8f3feef63c12541140c6c48dc2c6aa: Status 404 returned error can't find the container with id e9361580d826465fb864e6381de4525e2f8f3feef63c12541140c6c48dc2c6aa
Sep 11 14:06:57.275612 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:57.275571    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:06:57.275793 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:57.275726    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:06:57.275793 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:57.275768    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:06:58.209341 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:58.209303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j" event={"ID":"407aec10-e5ca-4004-84ca-7deecac5ef3b","Type":"ContainerStarted","Data":"330d41fcf6b7cfa22d44b39e979155e4c0c580b99b594ac837715e03525720f6"}
Sep 11 14:06:58.209761 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:58.209363    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j" event={"ID":"407aec10-e5ca-4004-84ca-7deecac5ef3b","Type":"ContainerStarted","Data":"e9361580d826465fb864e6381de4525e2f8f3feef63c12541140c6c48dc2c6aa"}
Sep 11 14:06:58.236920 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:58.236863    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j" podStartSLOduration=2.236841547 podStartE2EDuration="2.236841547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:06:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:58.233939945 +0000 UTC m=+1099.631282685" watchObservedRunningTime="2026-09-11 14:06:58.236841547 +0000 UTC m=+1099.634184283"
Sep 11 14:06:58.995816 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:58.995767    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:59.001313 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:59.001279    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:59.214516 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:59.214482    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:06:59.215357 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:06:59.215340    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:07:01.276323 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.276284    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"]
Sep 11 14:07:01.278882 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.278865    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:01.281100 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.281072    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-6b9470-serving-cert\""
Sep 11 14:07:01.289828 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.289718    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"]
Sep 11 14:07:01.320736 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.320696    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:01.320956 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.320743    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:01.320956 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.320769    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:01.320956 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.320834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2fqb\" (UniqueName: \"kubernetes.io/projected/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-kube-api-access-w2fqb\") pod \"maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:01.421622 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.421582    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:01.421622 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.421663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:01.421934 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.421685    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:01.421934 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.421725    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2fqb\" (UniqueName: \"kubernetes.io/projected/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-kube-api-access-w2fqb\") pod \"maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:01.422396 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.422369    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:01.424266 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.424245    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:01.424398 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.424279    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:01.430521 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.430491    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2fqb\" (UniqueName: \"kubernetes.io/projected/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-kube-api-access-w2fqb\") pod \"maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:01.593063 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.593011    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:01.867414 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:01.867382    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"]
Sep 11 14:07:01.870352 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:07:01.870325    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5df03b0_55f0_4bf9_a31e_be9a9e04218f.slice/crio-08ee012c423c5659bfed99b4ec0b8579924373ee27a2fc94cfd5c338b5ea3668 WatchSource:0}: Error finding container 08ee012c423c5659bfed99b4ec0b8579924373ee27a2fc94cfd5c338b5ea3668: Status 404 returned error can't find the container with id 08ee012c423c5659bfed99b4ec0b8579924373ee27a2fc94cfd5c338b5ea3668
Sep 11 14:07:02.229598 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:02.229508    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h" event={"ID":"a5df03b0-55f0-4bf9-a31e-be9a9e04218f","Type":"ContainerStarted","Data":"08ee012c423c5659bfed99b4ec0b8579924373ee27a2fc94cfd5c338b5ea3668"}
Sep 11 14:07:03.235336 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:03.235295    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h" event={"ID":"a5df03b0-55f0-4bf9-a31e-be9a9e04218f","Type":"ContainerStarted","Data":"35c3b73768224e20a61f0f4faa22014cc0d12824f36a5716312e29dafbf2507a"}
Sep 11 14:07:03.235788 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:03.235384    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:03.259836 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:03.259773    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h" podStartSLOduration=1.874806302 podStartE2EDuration="2.259755731s" totalImagesPullingTime="384.949429ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:07:03.258104993 +0000 UTC m=+1104.655447730" watchObservedRunningTime="2026-09-11 14:07:03.259755731 +0000 UTC m=+1104.657098467"
Sep 11 14:07:09.246100 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:09.246069    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:07:15.256929 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:15.256869    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 14:07:15.257385 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:15.256969    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j" podUID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 14:07:16.304783 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:16.304750    2563 generic.go:356] "Generic (PLEG): container finished" podID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerID="330d41fcf6b7cfa22d44b39e979155e4c0c580b99b594ac837715e03525720f6" exitCode=0
Sep 11 14:07:16.305223 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:16.304823    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j" event={"ID":"407aec10-e5ca-4004-84ca-7deecac5ef3b","Type":"ContainerDied","Data":"330d41fcf6b7cfa22d44b39e979155e4c0c580b99b594ac837715e03525720f6"}
Sep 11 14:07:16.305277 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:16.305249    2563 scope.go:117] "RemoveContainer" containerID="330d41fcf6b7cfa22d44b39e979155e4c0c580b99b594ac837715e03525720f6"
Sep 11 14:07:16.306351 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:16.306323    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:07:16.306406 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:16.306393    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:07:16.306450 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:16.306429    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:07:17.312043 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:17.311998    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j" event={"ID":"407aec10-e5ca-4004-84ca-7deecac5ef3b","Type":"ContainerStarted","Data":"6a6fc535466b82e873174c0a38edb0c33f712c72d3f01623f5fa8fc593e589c3"}
Sep 11 14:07:17.995393 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:17.995310    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:07:17.995841 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:17.995736    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 11 14:07:17.996058 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:17.996031    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j" podUID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 11 14:07:18.995482 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:18.995442    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 11 14:07:18.996103 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:18.996068    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j" podUID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 11 14:07:19.996070 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:19.996027    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 11 14:07:19.996727 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:19.996692    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j" podUID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 11 14:07:20.996594 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:20.996077    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 11 14:07:20.996594 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:20.996164    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j" podUID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 11 14:07:21.995589 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:21.995535    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 11 14:07:21.995846 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:21.995612    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j" podUID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 11 14:07:22.999123 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:22.999074    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:07:22.999502 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:22.999385    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:07:23.000286 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:23.000271    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:07:29.911674 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:29.911564    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"]
Sep 11 14:07:29.914453 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:29.914425    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:29.917217 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:29.917189    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-01faa9-serving-cert\""
Sep 11 14:07:29.928968 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:29.928922    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"]
Sep 11 14:07:30.008747 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:30.008696    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:30.008747 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:30.008769    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kd8t\" (UniqueName: \"kubernetes.io/projected/fa369955-8ed1-4633-9824-17621e4c6a71-kube-api-access-8kd8t\") pod \"maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:30.009038 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:30.008814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa369955-8ed1-4633-9824-17621e4c6a71-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:30.009038 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:30.008834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:30.109742 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:30.109701    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa369955-8ed1-4633-9824-17621e4c6a71-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:30.109742 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:30.109742    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:30.110020 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:30.109809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:30.110020 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:30.109853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kd8t\" (UniqueName: \"kubernetes.io/projected/fa369955-8ed1-4633-9824-17621e4c6a71-kube-api-access-8kd8t\") pod \"maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:30.110020 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:07:30.109934    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-serving-cert: secret "maas-api-e2e-shared-b-w2-01faa9-serving-cert" not found
Sep 11 14:07:30.110020 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:07:30.110020    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-maas-api-tls podName:fa369955-8ed1-4633-9824-17621e4c6a71 nodeName:}" failed. No retries permitted until 2026-09-11 14:07:30.609999673 +0000 UTC m=+1132.007342401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-maas-api-tls") pod "maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v" (UID: "fa369955-8ed1-4633-9824-17621e4c6a71") : secret "maas-api-e2e-shared-b-w2-01faa9-serving-cert" not found
Sep 11 14:07:30.110395 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:30.110368    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa369955-8ed1-4633-9824-17621e4c6a71-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:30.112490 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:30.112469    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:30.123125 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:30.123098    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kd8t\" (UniqueName: \"kubernetes.io/projected/fa369955-8ed1-4633-9824-17621e4c6a71-kube-api-access-8kd8t\") pod \"maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:30.615108 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:30.615064    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:30.615307 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:07:30.615222    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-serving-cert: secret "maas-api-e2e-shared-b-w2-01faa9-serving-cert" not found
Sep 11 14:07:30.615307 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:07:30.615304    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-maas-api-tls podName:fa369955-8ed1-4633-9824-17621e4c6a71 nodeName:}" failed. No retries permitted until 2026-09-11 14:07:31.61528624 +0000 UTC m=+1133.012628955 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-maas-api-tls") pod "maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v" (UID: "fa369955-8ed1-4633-9824-17621e4c6a71") : secret "maas-api-e2e-shared-b-w2-01faa9-serving-cert" not found
Sep 11 14:07:31.625189 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:31.625135    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:31.627843 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:31.627811    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:31.728782 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:31.728739    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:32.003620 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:32.003592    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"]
Sep 11 14:07:32.006102 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:07:32.006064    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa369955_8ed1_4633_9824_17621e4c6a71.slice/crio-07c9927ca943e2467542ffb5fee26d46dd62c95888f812260e18d41f6bfc4d2d WatchSource:0}: Error finding container 07c9927ca943e2467542ffb5fee26d46dd62c95888f812260e18d41f6bfc4d2d: Status 404 returned error can't find the container with id 07c9927ca943e2467542ffb5fee26d46dd62c95888f812260e18d41f6bfc4d2d
Sep 11 14:07:32.384198 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:32.384157    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v" event={"ID":"fa369955-8ed1-4633-9824-17621e4c6a71","Type":"ContainerStarted","Data":"07c9927ca943e2467542ffb5fee26d46dd62c95888f812260e18d41f6bfc4d2d"}
Sep 11 14:07:33.390720 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:33.390681    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v" event={"ID":"fa369955-8ed1-4633-9824-17621e4c6a71","Type":"ContainerStarted","Data":"dbe3e890670ef9af9794bf550b58d2f60b924312ec521bcc1a0a7759629eec88"}
Sep 11 14:07:33.391120 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:33.390823    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:07:33.408900 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:33.408839    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v" podStartSLOduration=3.946485502 podStartE2EDuration="4.408817952s" totalImagesPullingTime="462.33245ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:07:33.406850151 +0000 UTC m=+1134.804192888" watchObservedRunningTime="2026-09-11 14:07:33.408817952 +0000 UTC m=+1134.806160689"
Sep 11 14:07:39.401121 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:07:39.401089    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:08:09.140170 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.140132    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"]
Sep 11 14:08:09.142974 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.142951    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:08:09.145617 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.145595    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-fc83333c-serving-cert\""
Sep 11 14:08:09.155417 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.155382    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"]
Sep 11 14:08:09.278985 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.278946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n598\" (UniqueName: \"kubernetes.io/projected/34e427b9-91b8-493c-a89d-b659abe82bcc-kube-api-access-7n598\") pod \"maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:08:09.279203 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.279104    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34e427b9-91b8-493c-a89d-b659abe82bcc-openshift-service-ca\") pod \"maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:08:09.279203 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.279184    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34e427b9-91b8-493c-a89d-b659abe82bcc-maas-api-tls\") pod \"maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:08:09.279318 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.279223    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34e427b9-91b8-493c-a89d-b659abe82bcc-metrics-certs\") pod \"maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:08:09.379740 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.379711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n598\" (UniqueName: \"kubernetes.io/projected/34e427b9-91b8-493c-a89d-b659abe82bcc-kube-api-access-7n598\") pod \"maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:08:09.379907 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.379767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34e427b9-91b8-493c-a89d-b659abe82bcc-openshift-service-ca\") pod \"maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:08:09.379907 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.379797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34e427b9-91b8-493c-a89d-b659abe82bcc-maas-api-tls\") pod \"maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:08:09.379907 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.379818    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34e427b9-91b8-493c-a89d-b659abe82bcc-metrics-certs\") pod \"maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:08:09.380694 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.380589    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34e427b9-91b8-493c-a89d-b659abe82bcc-openshift-service-ca\") pod \"maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:08:09.382551 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.382521    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34e427b9-91b8-493c-a89d-b659abe82bcc-maas-api-tls\") pod \"maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:08:09.382551 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.382534    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34e427b9-91b8-493c-a89d-b659abe82bcc-metrics-certs\") pod \"maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:08:09.391077 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.391011    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n598\" (UniqueName: \"kubernetes.io/projected/34e427b9-91b8-493c-a89d-b659abe82bcc-kube-api-access-7n598\") pod \"maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:08:09.456689 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.456658    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:08:09.736609 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:09.736583    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"]
Sep 11 14:08:09.738877 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:08:09.738850    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34e427b9_91b8_493c_a89d_b659abe82bcc.slice/crio-e821e3da1f86a57b04b0aadc4642ed24b10423ee7cbc0849aed293e0f748dffb WatchSource:0}: Error finding container e821e3da1f86a57b04b0aadc4642ed24b10423ee7cbc0849aed293e0f748dffb: Status 404 returned error can't find the container with id e821e3da1f86a57b04b0aadc4642ed24b10423ee7cbc0849aed293e0f748dffb
Sep 11 14:08:10.575217 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:10.575178    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh" event={"ID":"34e427b9-91b8-493c-a89d-b659abe82bcc","Type":"ContainerStarted","Data":"f5a4b4c97921b3d74ab581350214e35f2ad9d95e2600ed8d8c2eaf21e5509f08"}
Sep 11 14:08:10.575689 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:10.575241    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:08:10.575689 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:10.575263    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh" event={"ID":"34e427b9-91b8-493c-a89d-b659abe82bcc","Type":"ContainerStarted","Data":"e821e3da1f86a57b04b0aadc4642ed24b10423ee7cbc0849aed293e0f748dffb"}
Sep 11 14:08:10.594416 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:10.594359    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh" podStartSLOduration=1.128382406 podStartE2EDuration="1.594336631s" totalImagesPullingTime="465.954225ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:08:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:08:10.592939198 +0000 UTC m=+1171.990281937" watchObservedRunningTime="2026-09-11 14:08:10.594336631 +0000 UTC m=+1171.991679373"
Sep 11 14:08:16.586187 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:08:16.586158    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:09:17.641436 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:17.641399    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"]
Sep 11 14:09:17.642026 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:17.641688    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh" podUID="34e427b9-91b8-493c-a89d-b659abe82bcc" containerName="maas-api" containerID="cri-o://f5a4b4c97921b3d74ab581350214e35f2ad9d95e2600ed8d8c2eaf21e5509f08" gracePeriod=30
Sep 11 14:09:17.906777 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:17.906744    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:09:17.917361 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:17.917330    2563 generic.go:356] "Generic (PLEG): container finished" podID="34e427b9-91b8-493c-a89d-b659abe82bcc" containerID="f5a4b4c97921b3d74ab581350214e35f2ad9d95e2600ed8d8c2eaf21e5509f08" exitCode=0
Sep 11 14:09:17.917555 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:17.917384    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"
Sep 11 14:09:17.917555 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:17.917393    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh" event={"ID":"34e427b9-91b8-493c-a89d-b659abe82bcc","Type":"ContainerDied","Data":"f5a4b4c97921b3d74ab581350214e35f2ad9d95e2600ed8d8c2eaf21e5509f08"}
Sep 11 14:09:17.917555 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:17.917436    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh" event={"ID":"34e427b9-91b8-493c-a89d-b659abe82bcc","Type":"ContainerDied","Data":"e821e3da1f86a57b04b0aadc4642ed24b10423ee7cbc0849aed293e0f748dffb"}
Sep 11 14:09:17.917555 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:17.917457    2563 scope.go:117] "RemoveContainer" containerID="f5a4b4c97921b3d74ab581350214e35f2ad9d95e2600ed8d8c2eaf21e5509f08"
Sep 11 14:09:17.928832 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:17.928801    2563 scope.go:117] "RemoveContainer" containerID="f5a4b4c97921b3d74ab581350214e35f2ad9d95e2600ed8d8c2eaf21e5509f08"
Sep 11 14:09:17.929267 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:09:17.929240    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5a4b4c97921b3d74ab581350214e35f2ad9d95e2600ed8d8c2eaf21e5509f08\": container with ID starting with f5a4b4c97921b3d74ab581350214e35f2ad9d95e2600ed8d8c2eaf21e5509f08 not found: ID does not exist" containerID="f5a4b4c97921b3d74ab581350214e35f2ad9d95e2600ed8d8c2eaf21e5509f08"
Sep 11 14:09:17.929358 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:17.929284    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5a4b4c97921b3d74ab581350214e35f2ad9d95e2600ed8d8c2eaf21e5509f08"} err="failed to get container status \"f5a4b4c97921b3d74ab581350214e35f2ad9d95e2600ed8d8c2eaf21e5509f08\": rpc error: code = NotFound desc = could not find container \"f5a4b4c97921b3d74ab581350214e35f2ad9d95e2600ed8d8c2eaf21e5509f08\": container with ID starting with f5a4b4c97921b3d74ab581350214e35f2ad9d95e2600ed8d8c2eaf21e5509f08 not found: ID does not exist"
Sep 11 14:09:18.017926 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:18.017880    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34e427b9-91b8-493c-a89d-b659abe82bcc-metrics-certs\") pod \"34e427b9-91b8-493c-a89d-b659abe82bcc\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") "
Sep 11 14:09:18.018122 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:18.017942    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34e427b9-91b8-493c-a89d-b659abe82bcc-maas-api-tls\") pod \"34e427b9-91b8-493c-a89d-b659abe82bcc\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") "
Sep 11 14:09:18.018122 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:18.017991    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34e427b9-91b8-493c-a89d-b659abe82bcc-openshift-service-ca\") pod \"34e427b9-91b8-493c-a89d-b659abe82bcc\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") "
Sep 11 14:09:18.018261 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:18.018148    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7n598\" (UniqueName: \"kubernetes.io/projected/34e427b9-91b8-493c-a89d-b659abe82bcc-kube-api-access-7n598\") pod \"34e427b9-91b8-493c-a89d-b659abe82bcc\" (UID: \"34e427b9-91b8-493c-a89d-b659abe82bcc\") "
Sep 11 14:09:18.018398 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:18.018371    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34e427b9-91b8-493c-a89d-b659abe82bcc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "34e427b9-91b8-493c-a89d-b659abe82bcc" (UID: "34e427b9-91b8-493c-a89d-b659abe82bcc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:09:18.020496 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:18.020459    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34e427b9-91b8-493c-a89d-b659abe82bcc-kube-api-access-7n598" (OuterVolumeSpecName: "kube-api-access-7n598") pod "34e427b9-91b8-493c-a89d-b659abe82bcc" (UID: "34e427b9-91b8-493c-a89d-b659abe82bcc"). InnerVolumeSpecName "kube-api-access-7n598". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:09:18.020808 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:18.020777    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34e427b9-91b8-493c-a89d-b659abe82bcc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "34e427b9-91b8-493c-a89d-b659abe82bcc" (UID: "34e427b9-91b8-493c-a89d-b659abe82bcc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:09:18.020808 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:18.020790    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34e427b9-91b8-493c-a89d-b659abe82bcc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "34e427b9-91b8-493c-a89d-b659abe82bcc" (UID: "34e427b9-91b8-493c-a89d-b659abe82bcc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:09:18.119415 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:18.119377    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34e427b9-91b8-493c-a89d-b659abe82bcc-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:09:18.119415 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:18.119408    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34e427b9-91b8-493c-a89d-b659abe82bcc-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:09:18.119648 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:18.119428    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34e427b9-91b8-493c-a89d-b659abe82bcc-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:09:18.119648 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:18.119437    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7n598\" (UniqueName: \"kubernetes.io/projected/34e427b9-91b8-493c-a89d-b659abe82bcc-kube-api-access-7n598\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:09:18.238232 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:18.238191    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"]
Sep 11 14:09:18.243397 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:18.243361    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fc83333c-dd9bcb75d-68nqh"]
Sep 11 14:09:19.131745 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:09:19.131713    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34e427b9-91b8-493c-a89d-b659abe82bcc" path="/var/lib/kubelet/pods/34e427b9-91b8-493c-a89d-b659abe82bcc/volumes"
Sep 11 14:10:47.700291 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.700253    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"]
Sep 11 14:10:47.701653 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.701601    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34e427b9-91b8-493c-a89d-b659abe82bcc" containerName="maas-api"
Sep 11 14:10:47.701653 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.701655    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="34e427b9-91b8-493c-a89d-b659abe82bcc" containerName="maas-api"
Sep 11 14:10:47.701851 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.701808    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="34e427b9-91b8-493c-a89d-b659abe82bcc" containerName="maas-api"
Sep 11 14:10:47.704585 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.704557    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.706912 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.706865    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-427751-openshift-default-dockercfg-v2clm\""
Sep 11 14:10:47.727680 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.727639    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"]
Sep 11 14:10:47.758277 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.758226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istiod-ca-cert\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.758480 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.758311    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-envoy\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.758480 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.758331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szjh7\" (UniqueName: \"kubernetes.io/projected/611e3397-2216-4f4b-a2f2-ce1db9f70b27-kube-api-access-szjh7\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.758480 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.758406    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-workload-socket\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.758480 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.758441    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-token\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.758727 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.758498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-workload-certs\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.758727 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.758519    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-podinfo\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.758727 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.758613    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-credential-socket\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.758727 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.758680    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-data\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.859797 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.859748    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istiod-ca-cert\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.860000 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.859852    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-envoy\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.860000 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.859878    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szjh7\" (UniqueName: \"kubernetes.io/projected/611e3397-2216-4f4b-a2f2-ce1db9f70b27-kube-api-access-szjh7\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.860000 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.859912    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-workload-socket\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.860000 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.859952    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-token\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.860236 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.860022    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-workload-certs\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.860236 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.860045    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-podinfo\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.860236 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.860080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-credential-socket\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.860236 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.860101    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-data\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.860690 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.860582    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-credential-socket\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.860690 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.860616    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istiod-ca-cert\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.860690 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.860619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-data\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.860690 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.860582    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-workload-socket\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.860690 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.860686    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-workload-certs\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.862600 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.862578    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-envoy\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.862747 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.862729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-podinfo\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.870124 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.870063    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-token\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:47.870280 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:47.870252    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szjh7\" (UniqueName: \"kubernetes.io/projected/611e3397-2216-4f4b-a2f2-ce1db9f70b27-kube-api-access-szjh7\") pod \"e2e-api-a-427751-openshift-default-ff96f6776-cnv5l\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") " pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:48.018027 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:48.017925    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:48.294253 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:48.294178    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"]
Sep 11 14:10:48.297118 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:10:48.297080    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod611e3397_2216_4f4b_a2f2_ce1db9f70b27.slice/crio-a9a2b8e2930b984d8ef0ecf984b4b4ad42133e3dce2d6f994a54c5a27f70d85a WatchSource:0}: Error finding container a9a2b8e2930b984d8ef0ecf984b4b4ad42133e3dce2d6f994a54c5a27f70d85a: Status 404 returned error can't find the container with id a9a2b8e2930b984d8ef0ecf984b4b4ad42133e3dce2d6f994a54c5a27f70d85a
Sep 11 14:10:48.299434 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:48.299397    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:10:48.299526 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:48.299468    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:10:48.299526 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:48.299498    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:10:48.346842 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:48.346808    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l" event={"ID":"611e3397-2216-4f4b-a2f2-ce1db9f70b27","Type":"ContainerStarted","Data":"a9a2b8e2930b984d8ef0ecf984b4b4ad42133e3dce2d6f994a54c5a27f70d85a"}
Sep 11 14:10:49.353641 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:49.353594    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l" event={"ID":"611e3397-2216-4f4b-a2f2-ce1db9f70b27","Type":"ContainerStarted","Data":"47117bcdd67616343cd66f770f84e2f1eb9fa8db36fe11e9fd03fed0f448386b"}
Sep 11 14:10:49.372907 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:49.372847    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l" podStartSLOduration=2.372826005 podStartE2EDuration="2.372826005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:10:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:10:49.372030229 +0000 UTC m=+1330.769372989" watchObservedRunningTime="2026-09-11 14:10:49.372826005 +0000 UTC m=+1330.770168741"
Sep 11 14:10:50.018888 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:50.018844    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:50.024651 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:50.024602    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:50.358717 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:50.358681    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:50.359807 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:50.359783    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:10:52.513961 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.513926    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"]
Sep 11 14:10:52.517156 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.517127    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:10:52.519374 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.519350    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-427751-serving-cert\""
Sep 11 14:10:52.525866 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.525771    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"]
Sep 11 14:10:52.602755 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.602716    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fph4k\" (UniqueName: \"kubernetes.io/projected/3a5c6774-2a35-484f-b3b8-5c828b663a73-kube-api-access-fph4k\") pod \"maas-api-e2e-api-a-427751-789fbc6bbf-p4zql\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:10:52.602755 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.602771    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a5c6774-2a35-484f-b3b8-5c828b663a73-openshift-service-ca\") pod \"maas-api-e2e-api-a-427751-789fbc6bbf-p4zql\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:10:52.602987 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.602833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a5c6774-2a35-484f-b3b8-5c828b663a73-maas-api-tls\") pod \"maas-api-e2e-api-a-427751-789fbc6bbf-p4zql\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:10:52.602987 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.602884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5c6774-2a35-484f-b3b8-5c828b663a73-metrics-certs\") pod \"maas-api-e2e-api-a-427751-789fbc6bbf-p4zql\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:10:52.703766 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.703714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a5c6774-2a35-484f-b3b8-5c828b663a73-maas-api-tls\") pod \"maas-api-e2e-api-a-427751-789fbc6bbf-p4zql\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:10:52.703988 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.703792    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5c6774-2a35-484f-b3b8-5c828b663a73-metrics-certs\") pod \"maas-api-e2e-api-a-427751-789fbc6bbf-p4zql\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:10:52.703988 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.703825    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fph4k\" (UniqueName: \"kubernetes.io/projected/3a5c6774-2a35-484f-b3b8-5c828b663a73-kube-api-access-fph4k\") pod \"maas-api-e2e-api-a-427751-789fbc6bbf-p4zql\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:10:52.703988 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.703859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a5c6774-2a35-484f-b3b8-5c828b663a73-openshift-service-ca\") pod \"maas-api-e2e-api-a-427751-789fbc6bbf-p4zql\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:10:52.704575 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.704553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a5c6774-2a35-484f-b3b8-5c828b663a73-openshift-service-ca\") pod \"maas-api-e2e-api-a-427751-789fbc6bbf-p4zql\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:10:52.706523 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.706489    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a5c6774-2a35-484f-b3b8-5c828b663a73-maas-api-tls\") pod \"maas-api-e2e-api-a-427751-789fbc6bbf-p4zql\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:10:52.706851 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.706831    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5c6774-2a35-484f-b3b8-5c828b663a73-metrics-certs\") pod \"maas-api-e2e-api-a-427751-789fbc6bbf-p4zql\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:10:52.711880 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.711846    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fph4k\" (UniqueName: \"kubernetes.io/projected/3a5c6774-2a35-484f-b3b8-5c828b663a73-kube-api-access-fph4k\") pod \"maas-api-e2e-api-a-427751-789fbc6bbf-p4zql\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:10:52.834835 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:52.834797    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:10:53.113840 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:53.113814    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"]
Sep 11 14:10:53.116360 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:10:53.116326    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5c6774_2a35_484f_b3b8_5c828b663a73.slice/crio-00f174777903956111aca437f1c187efad59e3a6f7d6c1e3d8013c924282ac80 WatchSource:0}: Error finding container 00f174777903956111aca437f1c187efad59e3a6f7d6c1e3d8013c924282ac80: Status 404 returned error can't find the container with id 00f174777903956111aca437f1c187efad59e3a6f7d6c1e3d8013c924282ac80
Sep 11 14:10:53.376185 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:53.376091    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql" event={"ID":"3a5c6774-2a35-484f-b3b8-5c828b663a73","Type":"ContainerStarted","Data":"00f174777903956111aca437f1c187efad59e3a6f7d6c1e3d8013c924282ac80"}
Sep 11 14:10:54.384065 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:54.384024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql" event={"ID":"3a5c6774-2a35-484f-b3b8-5c828b663a73","Type":"ContainerStarted","Data":"167b8d9daec7d4fe5e4d71a0379732e5d04d2acb7c893c744cc2074613d1561b"}
Sep 11 14:10:54.384495 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:54.384095    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:10:54.400384 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:10:54.400331    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql" podStartSLOduration=1.906108576 podStartE2EDuration="2.400313458s" totalImagesPullingTime="494.204882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:10:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:10:54.399285406 +0000 UTC m=+1335.796628144" watchObservedRunningTime="2026-09-11 14:10:54.400313458 +0000 UTC m=+1335.797656257"
Sep 11 14:11:00.394996 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:00.394916    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:11:12.475348 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:12.475310    2563 generic.go:356] "Generic (PLEG): container finished" podID="611e3397-2216-4f4b-a2f2-ce1db9f70b27" containerID="47117bcdd67616343cd66f770f84e2f1eb9fa8db36fe11e9fd03fed0f448386b" exitCode=0
Sep 11 14:11:12.475973 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:12.475382    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l" event={"ID":"611e3397-2216-4f4b-a2f2-ce1db9f70b27","Type":"ContainerDied","Data":"47117bcdd67616343cd66f770f84e2f1eb9fa8db36fe11e9fd03fed0f448386b"}
Sep 11 14:11:12.475973 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:12.475820    2563 scope.go:117] "RemoveContainer" containerID="47117bcdd67616343cd66f770f84e2f1eb9fa8db36fe11e9fd03fed0f448386b"
Sep 11 14:11:12.477133 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:12.477104    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:11:12.477230 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:12.477196    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:11:12.477269 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:12.477233    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:11:13.502365 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:13.502313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l" event={"ID":"611e3397-2216-4f4b-a2f2-ce1db9f70b27","Type":"ContainerStarted","Data":"7e91c712077a9350b772fa72810e420ad48e83a214284677307e7796e7e90f21"}
Sep 11 14:11:14.018391 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:14.018344    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:11:14.018611 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:14.018588    2563 patch_prober.go:28] interesting pod/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 11 14:11:14.018709 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:14.018673    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l" podUID="611e3397-2216-4f4b-a2f2-ce1db9f70b27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 11 14:11:15.019172 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:15.019130    2563 patch_prober.go:28] interesting pod/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 11 14:11:15.019599 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:15.019191    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l" podUID="611e3397-2216-4f4b-a2f2-ce1db9f70b27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 11 14:11:16.018723 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:16.018683    2563 patch_prober.go:28] interesting pod/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 11 14:11:16.018917 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:16.018739    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l" podUID="611e3397-2216-4f4b-a2f2-ce1db9f70b27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 11 14:11:17.019104 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:17.019062    2563 patch_prober.go:28] interesting pod/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 11 14:11:17.019515 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:17.019127    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l" podUID="611e3397-2216-4f4b-a2f2-ce1db9f70b27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 11 14:11:18.019098 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:18.019057    2563 patch_prober.go:28] interesting pod/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 11 14:11:18.019285 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:18.019117    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l" podUID="611e3397-2216-4f4b-a2f2-ce1db9f70b27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 11 14:11:19.023562 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:19.023522    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:11:19.024025 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:19.023835    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:11:19.024529 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:19.024511    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:11:21.440535 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.440493    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"]
Sep 11 14:11:21.443619 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.443593    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:11:21.446092 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.446066    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-1f2b49-serving-cert\""
Sep 11 14:11:21.458274 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.458248    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"]
Sep 11 14:11:21.481859 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.481826    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/768ae1b7-395b-434a-a100-fdca6a7a0343-openshift-service-ca\") pod \"maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:11:21.482084 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.481883    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw4f4\" (UniqueName: \"kubernetes.io/projected/768ae1b7-395b-434a-a100-fdca6a7a0343-kube-api-access-hw4f4\") pod \"maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:11:21.482084 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.481938    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768ae1b7-395b-434a-a100-fdca6a7a0343-maas-api-tls\") pod \"maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:11:21.482084 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.482034    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768ae1b7-395b-434a-a100-fdca6a7a0343-metrics-certs\") pod \"maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:11:21.583093 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.583048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw4f4\" (UniqueName: \"kubernetes.io/projected/768ae1b7-395b-434a-a100-fdca6a7a0343-kube-api-access-hw4f4\") pod \"maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:11:21.583093 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.583099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768ae1b7-395b-434a-a100-fdca6a7a0343-maas-api-tls\") pod \"maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:11:21.583308 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.583146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768ae1b7-395b-434a-a100-fdca6a7a0343-metrics-certs\") pod \"maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:11:21.583308 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.583187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/768ae1b7-395b-434a-a100-fdca6a7a0343-openshift-service-ca\") pod \"maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:11:21.583929 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.583902    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/768ae1b7-395b-434a-a100-fdca6a7a0343-openshift-service-ca\") pod \"maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:11:21.585852 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.585821    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768ae1b7-395b-434a-a100-fdca6a7a0343-metrics-certs\") pod \"maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:11:21.585852 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.585849    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768ae1b7-395b-434a-a100-fdca6a7a0343-maas-api-tls\") pod \"maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:11:21.594243 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.594216    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw4f4\" (UniqueName: \"kubernetes.io/projected/768ae1b7-395b-434a-a100-fdca6a7a0343-kube-api-access-hw4f4\") pod \"maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:11:21.755377 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:21.755275    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:11:22.042732 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:22.042112    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"]
Sep 11 14:11:22.049081 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:11:22.049045    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod768ae1b7_395b_434a_a100_fdca6a7a0343.slice/crio-eaf27a96d630d867de8d43660ccc11ef3091a326311d98cc6bd52130dc4eab24 WatchSource:0}: Error finding container eaf27a96d630d867de8d43660ccc11ef3091a326311d98cc6bd52130dc4eab24: Status 404 returned error can't find the container with id eaf27a96d630d867de8d43660ccc11ef3091a326311d98cc6bd52130dc4eab24
Sep 11 14:11:22.050916 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:22.050892    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:11:22.548331 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:22.548298    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c" event={"ID":"768ae1b7-395b-434a-a100-fdca6a7a0343","Type":"ContainerStarted","Data":"60a5b9bed28f51a3c46bf43c14d29d6b9320c91fc4ffb41e5fe58463ba6b3c92"}
Sep 11 14:11:22.548778 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:22.548348    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c" event={"ID":"768ae1b7-395b-434a-a100-fdca6a7a0343","Type":"ContainerStarted","Data":"eaf27a96d630d867de8d43660ccc11ef3091a326311d98cc6bd52130dc4eab24"}
Sep 11 14:11:22.548778 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:22.548445    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:11:22.612165 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:22.612050    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c" podStartSLOduration=1.315461539 podStartE2EDuration="1.612033776s" totalImagesPullingTime="296.572237ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:11:22.606411535 +0000 UTC m=+1364.003754272" watchObservedRunningTime="2026-09-11 14:11:22.612033776 +0000 UTC m=+1364.009376512"
Sep 11 14:11:28.559415 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:11:28.559382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:12:03.792255 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:03.792219    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"]
Sep 11 14:12:03.792878 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:03.792455    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql" podUID="3a5c6774-2a35-484f-b3b8-5c828b663a73" containerName="maas-api" containerID="cri-o://167b8d9daec7d4fe5e4d71a0379732e5d04d2acb7c893c744cc2074613d1561b" gracePeriod=30
Sep 11 14:12:04.048064 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.047996    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:12:04.095176 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.095139    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fph4k\" (UniqueName: \"kubernetes.io/projected/3a5c6774-2a35-484f-b3b8-5c828b663a73-kube-api-access-fph4k\") pod \"3a5c6774-2a35-484f-b3b8-5c828b663a73\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") "
Sep 11 14:12:04.095389 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.095227    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a5c6774-2a35-484f-b3b8-5c828b663a73-maas-api-tls\") pod \"3a5c6774-2a35-484f-b3b8-5c828b663a73\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") "
Sep 11 14:12:04.095389 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.095274    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a5c6774-2a35-484f-b3b8-5c828b663a73-openshift-service-ca\") pod \"3a5c6774-2a35-484f-b3b8-5c828b663a73\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") "
Sep 11 14:12:04.095389 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.095316    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5c6774-2a35-484f-b3b8-5c828b663a73-metrics-certs\") pod \"3a5c6774-2a35-484f-b3b8-5c828b663a73\" (UID: \"3a5c6774-2a35-484f-b3b8-5c828b663a73\") "
Sep 11 14:12:04.095790 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.095753    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a5c6774-2a35-484f-b3b8-5c828b663a73-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3a5c6774-2a35-484f-b3b8-5c828b663a73" (UID: "3a5c6774-2a35-484f-b3b8-5c828b663a73"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:12:04.097663 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.097608    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a5c6774-2a35-484f-b3b8-5c828b663a73-kube-api-access-fph4k" (OuterVolumeSpecName: "kube-api-access-fph4k") pod "3a5c6774-2a35-484f-b3b8-5c828b663a73" (UID: "3a5c6774-2a35-484f-b3b8-5c828b663a73"). InnerVolumeSpecName "kube-api-access-fph4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:12:04.098081 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.098059    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a5c6774-2a35-484f-b3b8-5c828b663a73-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3a5c6774-2a35-484f-b3b8-5c828b663a73" (UID: "3a5c6774-2a35-484f-b3b8-5c828b663a73"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:12:04.098176 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.098083    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a5c6774-2a35-484f-b3b8-5c828b663a73-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3a5c6774-2a35-484f-b3b8-5c828b663a73" (UID: "3a5c6774-2a35-484f-b3b8-5c828b663a73"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:12:04.197768 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.197673    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fph4k\" (UniqueName: \"kubernetes.io/projected/3a5c6774-2a35-484f-b3b8-5c828b663a73-kube-api-access-fph4k\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:04.197768 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.197709    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a5c6774-2a35-484f-b3b8-5c828b663a73-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:04.197768 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.197726    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a5c6774-2a35-484f-b3b8-5c828b663a73-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:04.197768 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.197740    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5c6774-2a35-484f-b3b8-5c828b663a73-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:04.755384 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.755348    2563 generic.go:356] "Generic (PLEG): container finished" podID="3a5c6774-2a35-484f-b3b8-5c828b663a73" containerID="167b8d9daec7d4fe5e4d71a0379732e5d04d2acb7c893c744cc2074613d1561b" exitCode=0
Sep 11 14:12:04.755384 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.755387    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"
Sep 11 14:12:04.755678 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.755443    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql" event={"ID":"3a5c6774-2a35-484f-b3b8-5c828b663a73","Type":"ContainerDied","Data":"167b8d9daec7d4fe5e4d71a0379732e5d04d2acb7c893c744cc2074613d1561b"}
Sep 11 14:12:04.755678 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.755498    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql" event={"ID":"3a5c6774-2a35-484f-b3b8-5c828b663a73","Type":"ContainerDied","Data":"00f174777903956111aca437f1c187efad59e3a6f7d6c1e3d8013c924282ac80"}
Sep 11 14:12:04.755678 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.755521    2563 scope.go:117] "RemoveContainer" containerID="167b8d9daec7d4fe5e4d71a0379732e5d04d2acb7c893c744cc2074613d1561b"
Sep 11 14:12:04.764972 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.764951    2563 scope.go:117] "RemoveContainer" containerID="167b8d9daec7d4fe5e4d71a0379732e5d04d2acb7c893c744cc2074613d1561b"
Sep 11 14:12:04.765307 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:12:04.765287    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"167b8d9daec7d4fe5e4d71a0379732e5d04d2acb7c893c744cc2074613d1561b\": container with ID starting with 167b8d9daec7d4fe5e4d71a0379732e5d04d2acb7c893c744cc2074613d1561b not found: ID does not exist" containerID="167b8d9daec7d4fe5e4d71a0379732e5d04d2acb7c893c744cc2074613d1561b"
Sep 11 14:12:04.765370 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.765316    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"167b8d9daec7d4fe5e4d71a0379732e5d04d2acb7c893c744cc2074613d1561b"} err="failed to get container status \"167b8d9daec7d4fe5e4d71a0379732e5d04d2acb7c893c744cc2074613d1561b\": rpc error: code = NotFound desc = could not find container \"167b8d9daec7d4fe5e4d71a0379732e5d04d2acb7c893c744cc2074613d1561b\": container with ID starting with 167b8d9daec7d4fe5e4d71a0379732e5d04d2acb7c893c744cc2074613d1561b not found: ID does not exist"
Sep 11 14:12:04.785011 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.784971    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"]
Sep 11 14:12:04.800330 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:04.800292    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-427751-789fbc6bbf-p4zql"]
Sep 11 14:12:05.130881 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:05.130847    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a5c6774-2a35-484f-b3b8-5c828b663a73" path="/var/lib/kubelet/pods/3a5c6774-2a35-484f-b3b8-5c828b663a73/volumes"
Sep 11 14:12:10.786136 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:10.786090    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"]
Sep 11 14:12:10.786792 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:10.786358    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l" podUID="611e3397-2216-4f4b-a2f2-ce1db9f70b27" containerName="istio-proxy" containerID="cri-o://7e91c712077a9350b772fa72810e420ad48e83a214284677307e7796e7e90f21" gracePeriod=30
Sep 11 14:12:16.044510 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.044479    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:12:16.120263 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120167    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-data\") pod \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") "
Sep 11 14:12:16.120263 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120218    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-workload-certs\") pod \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") "
Sep 11 14:12:16.120263 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120236    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-credential-socket\") pod \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") "
Sep 11 14:12:16.120544 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120279    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szjh7\" (UniqueName: \"kubernetes.io/projected/611e3397-2216-4f4b-a2f2-ce1db9f70b27-kube-api-access-szjh7\") pod \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") "
Sep 11 14:12:16.120544 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120299    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-workload-socket\") pod \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") "
Sep 11 14:12:16.120544 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120318    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-token\") pod \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") "
Sep 11 14:12:16.120544 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120348    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-envoy\") pod \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") "
Sep 11 14:12:16.120544 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120397    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-podinfo\") pod \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") "
Sep 11 14:12:16.120544 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120437    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istiod-ca-cert\") pod \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\" (UID: \"611e3397-2216-4f4b-a2f2-ce1db9f70b27\") "
Sep 11 14:12:16.120544 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120520    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-data" (OuterVolumeSpecName: "istio-data") pod "611e3397-2216-4f4b-a2f2-ce1db9f70b27" (UID: "611e3397-2216-4f4b-a2f2-ce1db9f70b27"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:16.120920 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120576    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "611e3397-2216-4f4b-a2f2-ce1db9f70b27" (UID: "611e3397-2216-4f4b-a2f2-ce1db9f70b27"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:16.120920 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120679    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "611e3397-2216-4f4b-a2f2-ce1db9f70b27" (UID: "611e3397-2216-4f4b-a2f2-ce1db9f70b27"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:16.120920 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120736    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "611e3397-2216-4f4b-a2f2-ce1db9f70b27" (UID: "611e3397-2216-4f4b-a2f2-ce1db9f70b27"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:16.120920 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120803    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-data\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:16.120920 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120817    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-workload-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:16.120920 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120825    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-credential-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:16.120920 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.120885    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "611e3397-2216-4f4b-a2f2-ce1db9f70b27" (UID: "611e3397-2216-4f4b-a2f2-ce1db9f70b27"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:12:16.122872 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.122842    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "611e3397-2216-4f4b-a2f2-ce1db9f70b27" (UID: "611e3397-2216-4f4b-a2f2-ce1db9f70b27"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:12:16.122999 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.122898    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/611e3397-2216-4f4b-a2f2-ce1db9f70b27-kube-api-access-szjh7" (OuterVolumeSpecName: "kube-api-access-szjh7") pod "611e3397-2216-4f4b-a2f2-ce1db9f70b27" (UID: "611e3397-2216-4f4b-a2f2-ce1db9f70b27"). InnerVolumeSpecName "kube-api-access-szjh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:12:16.123223 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.123201    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "611e3397-2216-4f4b-a2f2-ce1db9f70b27" (UID: "611e3397-2216-4f4b-a2f2-ce1db9f70b27"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:16.123268 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.123242    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-token" (OuterVolumeSpecName: "istio-token") pod "611e3397-2216-4f4b-a2f2-ce1db9f70b27" (UID: "611e3397-2216-4f4b-a2f2-ce1db9f70b27"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:12:16.221903 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.221866    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-podinfo\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:16.221903 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.221896    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istiod-ca-cert\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:16.221903 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.221906    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szjh7\" (UniqueName: \"kubernetes.io/projected/611e3397-2216-4f4b-a2f2-ce1db9f70b27-kube-api-access-szjh7\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:16.222146 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.221915    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-workload-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:16.222146 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.221924    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-token\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:16.222146 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.221932    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/611e3397-2216-4f4b-a2f2-ce1db9f70b27-istio-envoy\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:16.827422 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.827384    2563 generic.go:356] "Generic (PLEG): container finished" podID="611e3397-2216-4f4b-a2f2-ce1db9f70b27" containerID="7e91c712077a9350b772fa72810e420ad48e83a214284677307e7796e7e90f21" exitCode=0
Sep 11 14:12:16.827672 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.827452    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"
Sep 11 14:12:16.827672 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.827471    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l" event={"ID":"611e3397-2216-4f4b-a2f2-ce1db9f70b27","Type":"ContainerDied","Data":"7e91c712077a9350b772fa72810e420ad48e83a214284677307e7796e7e90f21"}
Sep 11 14:12:16.827672 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.827533    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l" event={"ID":"611e3397-2216-4f4b-a2f2-ce1db9f70b27","Type":"ContainerDied","Data":"a9a2b8e2930b984d8ef0ecf984b4b4ad42133e3dce2d6f994a54c5a27f70d85a"}
Sep 11 14:12:16.827672 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.827557    2563 scope.go:117] "RemoveContainer" containerID="7e91c712077a9350b772fa72810e420ad48e83a214284677307e7796e7e90f21"
Sep 11 14:12:16.837530 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.837509    2563 scope.go:117] "RemoveContainer" containerID="47117bcdd67616343cd66f770f84e2f1eb9fa8db36fe11e9fd03fed0f448386b"
Sep 11 14:12:16.846704 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.846680    2563 scope.go:117] "RemoveContainer" containerID="7e91c712077a9350b772fa72810e420ad48e83a214284677307e7796e7e90f21"
Sep 11 14:12:16.847049 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:12:16.847030    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e91c712077a9350b772fa72810e420ad48e83a214284677307e7796e7e90f21\": container with ID starting with 7e91c712077a9350b772fa72810e420ad48e83a214284677307e7796e7e90f21 not found: ID does not exist" containerID="7e91c712077a9350b772fa72810e420ad48e83a214284677307e7796e7e90f21"
Sep 11 14:12:16.847100 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.847062    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e91c712077a9350b772fa72810e420ad48e83a214284677307e7796e7e90f21"} err="failed to get container status \"7e91c712077a9350b772fa72810e420ad48e83a214284677307e7796e7e90f21\": rpc error: code = NotFound desc = could not find container \"7e91c712077a9350b772fa72810e420ad48e83a214284677307e7796e7e90f21\": container with ID starting with 7e91c712077a9350b772fa72810e420ad48e83a214284677307e7796e7e90f21 not found: ID does not exist"
Sep 11 14:12:16.847100 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.847081    2563 scope.go:117] "RemoveContainer" containerID="47117bcdd67616343cd66f770f84e2f1eb9fa8db36fe11e9fd03fed0f448386b"
Sep 11 14:12:16.847367 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:12:16.847348    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47117bcdd67616343cd66f770f84e2f1eb9fa8db36fe11e9fd03fed0f448386b\": container with ID starting with 47117bcdd67616343cd66f770f84e2f1eb9fa8db36fe11e9fd03fed0f448386b not found: ID does not exist" containerID="47117bcdd67616343cd66f770f84e2f1eb9fa8db36fe11e9fd03fed0f448386b"
Sep 11 14:12:16.847428 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.847371    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47117bcdd67616343cd66f770f84e2f1eb9fa8db36fe11e9fd03fed0f448386b"} err="failed to get container status \"47117bcdd67616343cd66f770f84e2f1eb9fa8db36fe11e9fd03fed0f448386b\": rpc error: code = NotFound desc = could not find container \"47117bcdd67616343cd66f770f84e2f1eb9fa8db36fe11e9fd03fed0f448386b\": container with ID starting with 47117bcdd67616343cd66f770f84e2f1eb9fa8db36fe11e9fd03fed0f448386b not found: ID does not exist"
Sep 11 14:12:16.857523 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.857477    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"]
Sep 11 14:12:16.862573 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:16.862543    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-427751-openshift-default-ff96f6776-cnv5l"]
Sep 11 14:12:17.131440 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:17.131362    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="611e3397-2216-4f4b-a2f2-ce1db9f70b27" path="/var/lib/kubelet/pods/611e3397-2216-4f4b-a2f2-ce1db9f70b27/volumes"
Sep 11 14:12:26.752996 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:26.752954    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"]
Sep 11 14:12:26.753547 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:26.753221    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c" podUID="768ae1b7-395b-434a-a100-fdca6a7a0343" containerName="maas-api" containerID="cri-o://60a5b9bed28f51a3c46bf43c14d29d6b9320c91fc4ffb41e5fe58463ba6b3c92" gracePeriod=30
Sep 11 14:12:26.878947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:26.878916    2563 generic.go:356] "Generic (PLEG): container finished" podID="768ae1b7-395b-434a-a100-fdca6a7a0343" containerID="60a5b9bed28f51a3c46bf43c14d29d6b9320c91fc4ffb41e5fe58463ba6b3c92" exitCode=0
Sep 11 14:12:26.879111 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:26.878990    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c" event={"ID":"768ae1b7-395b-434a-a100-fdca6a7a0343","Type":"ContainerDied","Data":"60a5b9bed28f51a3c46bf43c14d29d6b9320c91fc4ffb41e5fe58463ba6b3c92"}
Sep 11 14:12:27.007501 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.007425    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:12:27.022423 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.022377    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/768ae1b7-395b-434a-a100-fdca6a7a0343-openshift-service-ca\") pod \"768ae1b7-395b-434a-a100-fdca6a7a0343\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") "
Sep 11 14:12:27.022423 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.022429    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw4f4\" (UniqueName: \"kubernetes.io/projected/768ae1b7-395b-434a-a100-fdca6a7a0343-kube-api-access-hw4f4\") pod \"768ae1b7-395b-434a-a100-fdca6a7a0343\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") "
Sep 11 14:12:27.023142 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.022461    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768ae1b7-395b-434a-a100-fdca6a7a0343-metrics-certs\") pod \"768ae1b7-395b-434a-a100-fdca6a7a0343\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") "
Sep 11 14:12:27.023142 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.022557    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768ae1b7-395b-434a-a100-fdca6a7a0343-maas-api-tls\") pod \"768ae1b7-395b-434a-a100-fdca6a7a0343\" (UID: \"768ae1b7-395b-434a-a100-fdca6a7a0343\") "
Sep 11 14:12:27.023142 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.022878    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/768ae1b7-395b-434a-a100-fdca6a7a0343-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "768ae1b7-395b-434a-a100-fdca6a7a0343" (UID: "768ae1b7-395b-434a-a100-fdca6a7a0343"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:12:27.023142 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.022985    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/768ae1b7-395b-434a-a100-fdca6a7a0343-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:27.025830 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.025797    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/768ae1b7-395b-434a-a100-fdca6a7a0343-kube-api-access-hw4f4" (OuterVolumeSpecName: "kube-api-access-hw4f4") pod "768ae1b7-395b-434a-a100-fdca6a7a0343" (UID: "768ae1b7-395b-434a-a100-fdca6a7a0343"). InnerVolumeSpecName "kube-api-access-hw4f4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:12:27.026020 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.025983    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768ae1b7-395b-434a-a100-fdca6a7a0343-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "768ae1b7-395b-434a-a100-fdca6a7a0343" (UID: "768ae1b7-395b-434a-a100-fdca6a7a0343"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:12:27.026655 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.026433    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768ae1b7-395b-434a-a100-fdca6a7a0343-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "768ae1b7-395b-434a-a100-fdca6a7a0343" (UID: "768ae1b7-395b-434a-a100-fdca6a7a0343"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:12:27.124037 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.123991    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw4f4\" (UniqueName: \"kubernetes.io/projected/768ae1b7-395b-434a-a100-fdca6a7a0343-kube-api-access-hw4f4\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:27.124037 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.124032    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768ae1b7-395b-434a-a100-fdca6a7a0343-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:27.124037 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.124044    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768ae1b7-395b-434a-a100-fdca6a7a0343-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:27.885983 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.885954    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"
Sep 11 14:12:27.886385 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.885983    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c" event={"ID":"768ae1b7-395b-434a-a100-fdca6a7a0343","Type":"ContainerDied","Data":"eaf27a96d630d867de8d43660ccc11ef3091a326311d98cc6bd52130dc4eab24"}
Sep 11 14:12:27.886385 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.886057    2563 scope.go:117] "RemoveContainer" containerID="60a5b9bed28f51a3c46bf43c14d29d6b9320c91fc4ffb41e5fe58463ba6b3c92"
Sep 11 14:12:27.901895 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.901856    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"]
Sep 11 14:12:27.905025 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:27.904994    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1f2b49-76f46d5994-gv82c"]
Sep 11 14:12:29.131748 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:29.131698    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="768ae1b7-395b-434a-a100-fdca6a7a0343" path="/var/lib/kubelet/pods/768ae1b7-395b-434a-a100-fdca6a7a0343/volumes"
Sep 11 14:12:45.833328 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:45.833286    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"]
Sep 11 14:12:45.833761 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:45.833517    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h" podUID="a5df03b0-55f0-4bf9-a31e-be9a9e04218f" containerName="maas-api" containerID="cri-o://35c3b73768224e20a61f0f4faa22014cc0d12824f36a5716312e29dafbf2507a" gracePeriod=30
Sep 11 14:12:45.981259 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:45.981222    2563 generic.go:356] "Generic (PLEG): container finished" podID="a5df03b0-55f0-4bf9-a31e-be9a9e04218f" containerID="35c3b73768224e20a61f0f4faa22014cc0d12824f36a5716312e29dafbf2507a" exitCode=0
Sep 11 14:12:45.981470 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:45.981304    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h" event={"ID":"a5df03b0-55f0-4bf9-a31e-be9a9e04218f","Type":"ContainerDied","Data":"35c3b73768224e20a61f0f4faa22014cc0d12824f36a5716312e29dafbf2507a"}
Sep 11 14:12:46.088030 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.087962    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:12:46.195937 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.195899    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2fqb\" (UniqueName: \"kubernetes.io/projected/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-kube-api-access-w2fqb\") pod \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") "
Sep 11 14:12:46.196122 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.195963    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-metrics-certs\") pod \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") "
Sep 11 14:12:46.196122 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.196020    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-openshift-service-ca\") pod \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") "
Sep 11 14:12:46.196262 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.196119    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-maas-api-tls\") pod \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\" (UID: \"a5df03b0-55f0-4bf9-a31e-be9a9e04218f\") "
Sep 11 14:12:46.196400 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.196374    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a5df03b0-55f0-4bf9-a31e-be9a9e04218f" (UID: "a5df03b0-55f0-4bf9-a31e-be9a9e04218f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:12:46.198443 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.198404    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a5df03b0-55f0-4bf9-a31e-be9a9e04218f" (UID: "a5df03b0-55f0-4bf9-a31e-be9a9e04218f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:12:46.198443 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.198420    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a5df03b0-55f0-4bf9-a31e-be9a9e04218f" (UID: "a5df03b0-55f0-4bf9-a31e-be9a9e04218f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:12:46.198602 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.198520    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-kube-api-access-w2fqb" (OuterVolumeSpecName: "kube-api-access-w2fqb") pod "a5df03b0-55f0-4bf9-a31e-be9a9e04218f" (UID: "a5df03b0-55f0-4bf9-a31e-be9a9e04218f"). InnerVolumeSpecName "kube-api-access-w2fqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:12:46.296977 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.296932    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:46.296977 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.296970    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:46.296977 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.296980    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2fqb\" (UniqueName: \"kubernetes.io/projected/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-kube-api-access-w2fqb\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:46.296977 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.296988    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df03b0-55f0-4bf9-a31e-be9a9e04218f-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:46.988530 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.988501    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"
Sep 11 14:12:46.989045 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.988498    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h" event={"ID":"a5df03b0-55f0-4bf9-a31e-be9a9e04218f","Type":"ContainerDied","Data":"08ee012c423c5659bfed99b4ec0b8579924373ee27a2fc94cfd5c338b5ea3668"}
Sep 11 14:12:46.989045 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:46.988671    2563 scope.go:117] "RemoveContainer" containerID="35c3b73768224e20a61f0f4faa22014cc0d12824f36a5716312e29dafbf2507a"
Sep 11 14:12:47.010803 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:47.010767    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"]
Sep 11 14:12:47.014405 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:47.014369    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-6b9470-7d798b8bd8-n5l7h"]
Sep 11 14:12:47.131474 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:47.131440    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5df03b0-55f0-4bf9-a31e-be9a9e04218f" path="/var/lib/kubelet/pods/a5df03b0-55f0-4bf9-a31e-be9a9e04218f/volumes"
Sep 11 14:12:52.714390 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:52.714356    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"]
Sep 11 14:12:52.714837 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:52.714581    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j" podUID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerName="istio-proxy" containerID="cri-o://6a6fc535466b82e873174c0a38edb0c33f712c72d3f01623f5fa8fc593e589c3" gracePeriod=30
Sep 11 14:12:53.001790 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:53.001705    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 14:12:53.001790 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:53.001779    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j" podUID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 14:12:57.973598 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:57.973554    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:12:58.043013 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.042970    2563 generic.go:356] "Generic (PLEG): container finished" podID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerID="6a6fc535466b82e873174c0a38edb0c33f712c72d3f01623f5fa8fc593e589c3" exitCode=0
Sep 11 14:12:58.043177 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.043022    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j" event={"ID":"407aec10-e5ca-4004-84ca-7deecac5ef3b","Type":"ContainerDied","Data":"6a6fc535466b82e873174c0a38edb0c33f712c72d3f01623f5fa8fc593e589c3"}
Sep 11 14:12:58.043177 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.043086    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j" event={"ID":"407aec10-e5ca-4004-84ca-7deecac5ef3b","Type":"ContainerDied","Data":"e9361580d826465fb864e6381de4525e2f8f3feef63c12541140c6c48dc2c6aa"}
Sep 11 14:12:58.043177 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.043116    2563 scope.go:117] "RemoveContainer" containerID="6a6fc535466b82e873174c0a38edb0c33f712c72d3f01623f5fa8fc593e589c3"
Sep 11 14:12:58.043357 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.043030    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"
Sep 11 14:12:58.051933 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.051909    2563 scope.go:117] "RemoveContainer" containerID="330d41fcf6b7cfa22d44b39e979155e4c0c580b99b594ac837715e03525720f6"
Sep 11 14:12:58.060597 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.060574    2563 scope.go:117] "RemoveContainer" containerID="6a6fc535466b82e873174c0a38edb0c33f712c72d3f01623f5fa8fc593e589c3"
Sep 11 14:12:58.060985 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:12:58.060962    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a6fc535466b82e873174c0a38edb0c33f712c72d3f01623f5fa8fc593e589c3\": container with ID starting with 6a6fc535466b82e873174c0a38edb0c33f712c72d3f01623f5fa8fc593e589c3 not found: ID does not exist" containerID="6a6fc535466b82e873174c0a38edb0c33f712c72d3f01623f5fa8fc593e589c3"
Sep 11 14:12:58.061091 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.060992    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a6fc535466b82e873174c0a38edb0c33f712c72d3f01623f5fa8fc593e589c3"} err="failed to get container status \"6a6fc535466b82e873174c0a38edb0c33f712c72d3f01623f5fa8fc593e589c3\": rpc error: code = NotFound desc = could not find container \"6a6fc535466b82e873174c0a38edb0c33f712c72d3f01623f5fa8fc593e589c3\": container with ID starting with 6a6fc535466b82e873174c0a38edb0c33f712c72d3f01623f5fa8fc593e589c3 not found: ID does not exist"
Sep 11 14:12:58.061091 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.061011    2563 scope.go:117] "RemoveContainer" containerID="330d41fcf6b7cfa22d44b39e979155e4c0c580b99b594ac837715e03525720f6"
Sep 11 14:12:58.061319 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:12:58.061301    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"330d41fcf6b7cfa22d44b39e979155e4c0c580b99b594ac837715e03525720f6\": container with ID starting with 330d41fcf6b7cfa22d44b39e979155e4c0c580b99b594ac837715e03525720f6 not found: ID does not exist" containerID="330d41fcf6b7cfa22d44b39e979155e4c0c580b99b594ac837715e03525720f6"
Sep 11 14:12:58.061371 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.061327    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"330d41fcf6b7cfa22d44b39e979155e4c0c580b99b594ac837715e03525720f6"} err="failed to get container status \"330d41fcf6b7cfa22d44b39e979155e4c0c580b99b594ac837715e03525720f6\": rpc error: code = NotFound desc = could not find container \"330d41fcf6b7cfa22d44b39e979155e4c0c580b99b594ac837715e03525720f6\": container with ID starting with 330d41fcf6b7cfa22d44b39e979155e4c0c580b99b594ac837715e03525720f6 not found: ID does not exist"
Sep 11 14:12:58.106726 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.106690    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-envoy\") pod \"407aec10-e5ca-4004-84ca-7deecac5ef3b\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") "
Sep 11 14:12:58.106928 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.106749    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-podinfo\") pod \"407aec10-e5ca-4004-84ca-7deecac5ef3b\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") "
Sep 11 14:12:58.106928 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.106778    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7l9p\" (UniqueName: \"kubernetes.io/projected/407aec10-e5ca-4004-84ca-7deecac5ef3b-kube-api-access-b7l9p\") pod \"407aec10-e5ca-4004-84ca-7deecac5ef3b\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") "
Sep 11 14:12:58.106928 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.106799    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-data\") pod \"407aec10-e5ca-4004-84ca-7deecac5ef3b\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") "
Sep 11 14:12:58.106928 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.106826    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-workload-certs\") pod \"407aec10-e5ca-4004-84ca-7deecac5ef3b\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") "
Sep 11 14:12:58.106928 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.106893    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-credential-socket\") pod \"407aec10-e5ca-4004-84ca-7deecac5ef3b\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") "
Sep 11 14:12:58.106928 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.106920    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-token\") pod \"407aec10-e5ca-4004-84ca-7deecac5ef3b\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") "
Sep 11 14:12:58.107255 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.106976    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-workload-socket\") pod \"407aec10-e5ca-4004-84ca-7deecac5ef3b\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") "
Sep 11 14:12:58.107255 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.107057    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/407aec10-e5ca-4004-84ca-7deecac5ef3b-istiod-ca-cert\") pod \"407aec10-e5ca-4004-84ca-7deecac5ef3b\" (UID: \"407aec10-e5ca-4004-84ca-7deecac5ef3b\") "
Sep 11 14:12:58.107255 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.107126    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-data" (OuterVolumeSpecName: "istio-data") pod "407aec10-e5ca-4004-84ca-7deecac5ef3b" (UID: "407aec10-e5ca-4004-84ca-7deecac5ef3b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:58.107416 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.107346    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "407aec10-e5ca-4004-84ca-7deecac5ef3b" (UID: "407aec10-e5ca-4004-84ca-7deecac5ef3b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:58.107416 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.107356    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "407aec10-e5ca-4004-84ca-7deecac5ef3b" (UID: "407aec10-e5ca-4004-84ca-7deecac5ef3b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:58.107416 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.107405    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-data\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:58.107755 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.107461    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "407aec10-e5ca-4004-84ca-7deecac5ef3b" (UID: "407aec10-e5ca-4004-84ca-7deecac5ef3b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:58.107944 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.107918    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/407aec10-e5ca-4004-84ca-7deecac5ef3b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "407aec10-e5ca-4004-84ca-7deecac5ef3b" (UID: "407aec10-e5ca-4004-84ca-7deecac5ef3b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:12:58.109329 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.109295    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "407aec10-e5ca-4004-84ca-7deecac5ef3b" (UID: "407aec10-e5ca-4004-84ca-7deecac5ef3b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:12:58.109329 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.109318    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "407aec10-e5ca-4004-84ca-7deecac5ef3b" (UID: "407aec10-e5ca-4004-84ca-7deecac5ef3b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:12:58.109496 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.109337    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-token" (OuterVolumeSpecName: "istio-token") pod "407aec10-e5ca-4004-84ca-7deecac5ef3b" (UID: "407aec10-e5ca-4004-84ca-7deecac5ef3b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:12:58.109496 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.109348    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/407aec10-e5ca-4004-84ca-7deecac5ef3b-kube-api-access-b7l9p" (OuterVolumeSpecName: "kube-api-access-b7l9p") pod "407aec10-e5ca-4004-84ca-7deecac5ef3b" (UID: "407aec10-e5ca-4004-84ca-7deecac5ef3b"). InnerVolumeSpecName "kube-api-access-b7l9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:12:58.208781 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.208744    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-podinfo\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:58.208781 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.208773    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7l9p\" (UniqueName: \"kubernetes.io/projected/407aec10-e5ca-4004-84ca-7deecac5ef3b-kube-api-access-b7l9p\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:58.208781 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.208783    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-workload-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:58.208781 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.208793    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-credential-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:58.209072 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.208804    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-token\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:58.209072 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.208812    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-workload-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:58.209072 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.208820    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/407aec10-e5ca-4004-84ca-7deecac5ef3b-istiod-ca-cert\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:58.209072 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.208829    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/407aec10-e5ca-4004-84ca-7deecac5ef3b-istio-envoy\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:12:58.364361 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.364323    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"]
Sep 11 14:12:58.368840 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:58.368808    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b9470-openshift-default-7b4dd98bdb-92z6j"]
Sep 11 14:12:59.132042 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:12:59.132009    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="407aec10-e5ca-4004-84ca-7deecac5ef3b" path="/var/lib/kubelet/pods/407aec10-e5ca-4004-84ca-7deecac5ef3b/volumes"
Sep 11 14:13:03.978237 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:03.978196    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"]
Sep 11 14:13:03.978726 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:03.978479    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v" podUID="fa369955-8ed1-4633-9824-17621e4c6a71" containerName="maas-api" containerID="cri-o://dbe3e890670ef9af9794bf550b58d2f60b924312ec521bcc1a0a7759629eec88" gracePeriod=30
Sep 11 14:13:04.238049 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:04.237972    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:13:04.365743 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:04.365705    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-maas-api-tls\") pod \"fa369955-8ed1-4633-9824-17621e4c6a71\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") "
Sep 11 14:13:04.365956 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:04.365758    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kd8t\" (UniqueName: \"kubernetes.io/projected/fa369955-8ed1-4633-9824-17621e4c6a71-kube-api-access-8kd8t\") pod \"fa369955-8ed1-4633-9824-17621e4c6a71\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") "
Sep 11 14:13:04.365956 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:04.365793    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-metrics-certs\") pod \"fa369955-8ed1-4633-9824-17621e4c6a71\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") "
Sep 11 14:13:04.365956 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:04.365895    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa369955-8ed1-4633-9824-17621e4c6a71-openshift-service-ca\") pod \"fa369955-8ed1-4633-9824-17621e4c6a71\" (UID: \"fa369955-8ed1-4633-9824-17621e4c6a71\") "
Sep 11 14:13:04.366499 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:04.366466    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa369955-8ed1-4633-9824-17621e4c6a71-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fa369955-8ed1-4633-9824-17621e4c6a71" (UID: "fa369955-8ed1-4633-9824-17621e4c6a71"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:13:04.368696 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:04.368663    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa369955-8ed1-4633-9824-17621e4c6a71-kube-api-access-8kd8t" (OuterVolumeSpecName: "kube-api-access-8kd8t") pod "fa369955-8ed1-4633-9824-17621e4c6a71" (UID: "fa369955-8ed1-4633-9824-17621e4c6a71"). InnerVolumeSpecName "kube-api-access-8kd8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:13:04.368864 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:04.368757    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fa369955-8ed1-4633-9824-17621e4c6a71" (UID: "fa369955-8ed1-4633-9824-17621e4c6a71"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:13:04.369139 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:04.369106    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fa369955-8ed1-4633-9824-17621e4c6a71" (UID: "fa369955-8ed1-4633-9824-17621e4c6a71"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:13:04.466955 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:04.466911    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa369955-8ed1-4633-9824-17621e4c6a71-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:13:04.466955 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:04.466945    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:13:04.466955 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:04.466957    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kd8t\" (UniqueName: \"kubernetes.io/projected/fa369955-8ed1-4633-9824-17621e4c6a71-kube-api-access-8kd8t\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:13:04.466955 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:04.466965    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa369955-8ed1-4633-9824-17621e4c6a71-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:13:05.078483 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:05.078443    2563 generic.go:356] "Generic (PLEG): container finished" podID="fa369955-8ed1-4633-9824-17621e4c6a71" containerID="dbe3e890670ef9af9794bf550b58d2f60b924312ec521bcc1a0a7759629eec88" exitCode=0
Sep 11 14:13:05.078483 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:05.078483    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"
Sep 11 14:13:05.078980 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:05.078523    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v" event={"ID":"fa369955-8ed1-4633-9824-17621e4c6a71","Type":"ContainerDied","Data":"dbe3e890670ef9af9794bf550b58d2f60b924312ec521bcc1a0a7759629eec88"}
Sep 11 14:13:05.078980 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:05.078582    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v" event={"ID":"fa369955-8ed1-4633-9824-17621e4c6a71","Type":"ContainerDied","Data":"07c9927ca943e2467542ffb5fee26d46dd62c95888f812260e18d41f6bfc4d2d"}
Sep 11 14:13:05.078980 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:05.078611    2563 scope.go:117] "RemoveContainer" containerID="dbe3e890670ef9af9794bf550b58d2f60b924312ec521bcc1a0a7759629eec88"
Sep 11 14:13:05.087810 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:05.087788    2563 scope.go:117] "RemoveContainer" containerID="dbe3e890670ef9af9794bf550b58d2f60b924312ec521bcc1a0a7759629eec88"
Sep 11 14:13:05.088161 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:13:05.088143    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbe3e890670ef9af9794bf550b58d2f60b924312ec521bcc1a0a7759629eec88\": container with ID starting with dbe3e890670ef9af9794bf550b58d2f60b924312ec521bcc1a0a7759629eec88 not found: ID does not exist" containerID="dbe3e890670ef9af9794bf550b58d2f60b924312ec521bcc1a0a7759629eec88"
Sep 11 14:13:05.088221 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:05.088171    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbe3e890670ef9af9794bf550b58d2f60b924312ec521bcc1a0a7759629eec88"} err="failed to get container status \"dbe3e890670ef9af9794bf550b58d2f60b924312ec521bcc1a0a7759629eec88\": rpc error: code = NotFound desc = could not find container \"dbe3e890670ef9af9794bf550b58d2f60b924312ec521bcc1a0a7759629eec88\": container with ID starting with dbe3e890670ef9af9794bf550b58d2f60b924312ec521bcc1a0a7759629eec88 not found: ID does not exist"
Sep 11 14:13:05.099303 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:05.099270    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"]
Sep 11 14:13:05.104748 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:05.104713    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01faa9-5d6db5c96f-mmp7v"]
Sep 11 14:13:05.133005 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:05.132974    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa369955-8ed1-4633-9824-17621e4c6a71" path="/var/lib/kubelet/pods/fa369955-8ed1-4633-9824-17621e4c6a71/volumes"
Sep 11 14:13:23.016450 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016410    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"]
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016792    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerName="istio-proxy"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016804    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerName="istio-proxy"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016812    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa369955-8ed1-4633-9824-17621e4c6a71" containerName="maas-api"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016818    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa369955-8ed1-4633-9824-17621e4c6a71" containerName="maas-api"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016824    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerName="istio-proxy"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016829    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerName="istio-proxy"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016835    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a5c6774-2a35-484f-b3b8-5c828b663a73" containerName="maas-api"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016840    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a5c6774-2a35-484f-b3b8-5c828b663a73" containerName="maas-api"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016846    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="611e3397-2216-4f4b-a2f2-ce1db9f70b27" containerName="istio-proxy"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016851    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="611e3397-2216-4f4b-a2f2-ce1db9f70b27" containerName="istio-proxy"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016859    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="611e3397-2216-4f4b-a2f2-ce1db9f70b27" containerName="istio-proxy"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016864    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="611e3397-2216-4f4b-a2f2-ce1db9f70b27" containerName="istio-proxy"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016875    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5df03b0-55f0-4bf9-a31e-be9a9e04218f" containerName="maas-api"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016880    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5df03b0-55f0-4bf9-a31e-be9a9e04218f" containerName="maas-api"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016884    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="768ae1b7-395b-434a-a100-fdca6a7a0343" containerName="maas-api"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016889    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="768ae1b7-395b-434a-a100-fdca6a7a0343" containerName="maas-api"
Sep 11 14:13:23.016947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016954    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerName="istio-proxy"
Sep 11 14:13:23.017487 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016963    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a5c6774-2a35-484f-b3b8-5c828b663a73" containerName="maas-api"
Sep 11 14:13:23.017487 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016971    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="611e3397-2216-4f4b-a2f2-ce1db9f70b27" containerName="istio-proxy"
Sep 11 14:13:23.017487 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016976    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="407aec10-e5ca-4004-84ca-7deecac5ef3b" containerName="istio-proxy"
Sep 11 14:13:23.017487 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016981    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5df03b0-55f0-4bf9-a31e-be9a9e04218f" containerName="maas-api"
Sep 11 14:13:23.017487 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016986    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="768ae1b7-395b-434a-a100-fdca6a7a0343" containerName="maas-api"
Sep 11 14:13:23.017487 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016992    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa369955-8ed1-4633-9824-17621e4c6a71" containerName="maas-api"
Sep 11 14:13:23.017487 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.016997    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="611e3397-2216-4f4b-a2f2-ce1db9f70b27" containerName="istio-proxy"
Sep 11 14:13:23.020254 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.020229    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.022688 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.022617    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-e3622e-openshift-default-dockercfg-s4wv8\""
Sep 11 14:13:23.033455 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.033418    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"]
Sep 11 14:13:23.130065 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.130027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-envoy\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.130346 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.130326    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-workload-socket\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.130530 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.130508    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istiod-ca-cert\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.130716 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.130569    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-podinfo\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.130716 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.130598    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-data\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.130716 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.130665    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rbp5\" (UniqueName: \"kubernetes.io/projected/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-kube-api-access-7rbp5\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.130716 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.130695    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-token\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.130959 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.130753    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-credential-socket\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.130959 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.130778    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-workload-certs\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.232204 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.232158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-envoy\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.232413 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.232216    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-workload-socket\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.232413 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.232246    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istiod-ca-cert\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.232413 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.232287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-podinfo\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.232413 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.232312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-data\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.232413 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.232349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rbp5\" (UniqueName: \"kubernetes.io/projected/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-kube-api-access-7rbp5\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.232413 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.232377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-token\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.232768 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.232424    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-credential-socket\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.232768 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.232448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-workload-certs\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.232930 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.232897    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-workload-socket\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.233079 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.232974    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-data\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.233146 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.233086    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istiod-ca-cert\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.233202 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.233028    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-credential-socket\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.233286 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.233194    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-workload-certs\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.234754 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.234722    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-envoy\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.234991 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.234973    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-podinfo\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.241379 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.241337    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-token\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.241529 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.241448    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rbp5\" (UniqueName: \"kubernetes.io/projected/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-kube-api-access-7rbp5\") pod \"e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") " pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.333949 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.333908    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:23.611847 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.611815    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"]
Sep 11 14:13:23.614644 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:13:23.614599    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aeb9f97_b9f8_4bf3_b26e_3c856609127b.slice/crio-6f4616fcc28192dc6d4395289bfacddacb92b78e1ce509d6a54f0533b4d8816d WatchSource:0}: Error finding container 6f4616fcc28192dc6d4395289bfacddacb92b78e1ce509d6a54f0533b4d8816d: Status 404 returned error can't find the container with id 6f4616fcc28192dc6d4395289bfacddacb92b78e1ce509d6a54f0533b4d8816d
Sep 11 14:13:23.616894 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.616854    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:13:23.616996 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.616927    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:13:23.616996 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:23.616961    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:13:24.173553 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:24.173509    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" event={"ID":"0aeb9f97-b9f8-4bf3-b26e-3c856609127b","Type":"ContainerStarted","Data":"fd1c687e5d0ecec4f072394234701444e9ad789fe40381b60ee1d1bdbe8a7aad"}
Sep 11 14:13:24.174083 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:24.173578    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" event={"ID":"0aeb9f97-b9f8-4bf3-b26e-3c856609127b","Type":"ContainerStarted","Data":"6f4616fcc28192dc6d4395289bfacddacb92b78e1ce509d6a54f0533b4d8816d"}
Sep 11 14:13:24.197068 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:24.197001    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" podStartSLOduration=2.19697796 podStartE2EDuration="2.19697796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:24.195985497 +0000 UTC m=+1485.593328244" watchObservedRunningTime="2026-09-11 14:13:24.19697796 +0000 UTC m=+1485.594320699"
Sep 11 14:13:24.334725 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:24.334675    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:24.336565 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:24.336538    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 11 14:13:24.336707 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:24.336608    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 11 14:13:25.339563 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:25.339522    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:26.184357 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:26.184319    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:26.185347 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:26.185327    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:27.242075 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.242009    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"]
Sep 11 14:13:27.245934 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.245907    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:27.249577 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.249546    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-e3622e-serving-cert\""
Sep 11 14:13:27.264428 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.264378    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"]
Sep 11 14:13:27.367902 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.367845    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa54269d-cf0a-4259-a102-bd4a76557a4c-metrics-certs\") pod \"maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:27.368105 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.367922    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa54269d-cf0a-4259-a102-bd4a76557a4c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:27.368105 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.367953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm56n\" (UniqueName: \"kubernetes.io/projected/aa54269d-cf0a-4259-a102-bd4a76557a4c-kube-api-access-sm56n\") pod \"maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:27.368105 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.368067    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa54269d-cf0a-4259-a102-bd4a76557a4c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:27.469125 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.469080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa54269d-cf0a-4259-a102-bd4a76557a4c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:27.469329 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.469144    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa54269d-cf0a-4259-a102-bd4a76557a4c-metrics-certs\") pod \"maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:27.469329 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.469209    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa54269d-cf0a-4259-a102-bd4a76557a4c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:27.469329 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.469235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm56n\" (UniqueName: \"kubernetes.io/projected/aa54269d-cf0a-4259-a102-bd4a76557a4c-kube-api-access-sm56n\") pod \"maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:27.470238 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.470170    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa54269d-cf0a-4259-a102-bd4a76557a4c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:27.473078 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.473047    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa54269d-cf0a-4259-a102-bd4a76557a4c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:27.473242 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.473060    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa54269d-cf0a-4259-a102-bd4a76557a4c-metrics-certs\") pod \"maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:27.478466 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.478421    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm56n\" (UniqueName: \"kubernetes.io/projected/aa54269d-cf0a-4259-a102-bd4a76557a4c-kube-api-access-sm56n\") pod \"maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:27.558527 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.558481    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:27.848845 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:27.848815    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"]
Sep 11 14:13:27.850686 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:13:27.850655    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa54269d_cf0a_4259_a102_bd4a76557a4c.slice/crio-08287daf33725275e67c78c05960a5c5a61e1f231f6cd693ac32320b3111ea1c WatchSource:0}: Error finding container 08287daf33725275e67c78c05960a5c5a61e1f231f6cd693ac32320b3111ea1c: Status 404 returned error can't find the container with id 08287daf33725275e67c78c05960a5c5a61e1f231f6cd693ac32320b3111ea1c
Sep 11 14:13:28.195275 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:28.195200    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw" event={"ID":"aa54269d-cf0a-4259-a102-bd4a76557a4c","Type":"ContainerStarted","Data":"08287daf33725275e67c78c05960a5c5a61e1f231f6cd693ac32320b3111ea1c"}
Sep 11 14:13:29.201337 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:29.201297    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw" event={"ID":"aa54269d-cf0a-4259-a102-bd4a76557a4c","Type":"ContainerStarted","Data":"dda7c4920016710945137c2afffa086a5cd7bb462b5e3744ec5a772c79fae0d5"}
Sep 11 14:13:29.201846 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:29.201363    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:29.222071 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:29.222018    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw" podStartSLOduration=1.855329088 podStartE2EDuration="2.221997101s" totalImagesPullingTime="366.668013ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:29.220001356 +0000 UTC m=+1490.617344106" watchObservedRunningTime="2026-09-11 14:13:29.221997101 +0000 UTC m=+1490.619339838"
Sep 11 14:13:35.214457 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:35.214426    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:13:42.226890 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:42.226842    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 14:13:42.227295 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:42.226944    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 14:13:44.276780 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:44.276745    2563 generic.go:356] "Generic (PLEG): container finished" podID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerID="fd1c687e5d0ecec4f072394234701444e9ad789fe40381b60ee1d1bdbe8a7aad" exitCode=0
Sep 11 14:13:44.277284 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:44.276793    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" event={"ID":"0aeb9f97-b9f8-4bf3-b26e-3c856609127b","Type":"ContainerDied","Data":"fd1c687e5d0ecec4f072394234701444e9ad789fe40381b60ee1d1bdbe8a7aad"}
Sep 11 14:13:44.277284 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:44.277179    2563 scope.go:117] "RemoveContainer" containerID="fd1c687e5d0ecec4f072394234701444e9ad789fe40381b60ee1d1bdbe8a7aad"
Sep 11 14:13:44.278393 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:44.278356    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:13:44.278459 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:44.278432    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:13:44.278496 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:44.278463    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:13:45.287278 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:45.287238    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" event={"ID":"0aeb9f97-b9f8-4bf3-b26e-3c856609127b","Type":"ContainerStarted","Data":"4df63536b0dc9496f08dc243608be44db8f32e9be14cfa80644caffb2532862e"}
Sep 11 14:13:45.334867 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:45.334819    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:45.335178 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:45.335153    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 11 14:13:45.335280 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:45.335222    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 11 14:13:46.335112 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:46.335068    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 11 14:13:46.335515 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:46.335135    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 11 14:13:47.334353 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:47.334312    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 11 14:13:47.334543 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:47.334373    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 11 14:13:48.334893 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:48.334848    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 11 14:13:48.335332 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:48.334943    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 11 14:13:49.334986 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:49.334942    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 11 14:13:49.335400 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:49.335007    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 11 14:13:50.335286 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:50.335247    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 11 14:13:50.335737 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:50.335321    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 11 14:13:51.338794 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:51.338762    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:51.339174 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:51.339086    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:51.340007 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:51.339985    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:13:56.581840 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.581805    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"]
Sep 11 14:13:56.585438 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.585413    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:13:56.587720 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.587685    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-68d0c7-serving-cert\""
Sep 11 14:13:56.597056 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.597024    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"]
Sep 11 14:13:56.750098 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.750050    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:13:56.750475 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.750440    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-maas-api-tls\") pod \"maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:13:56.750597 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.750479    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-metrics-certs\") pod \"maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:13:56.750597 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.750558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhwxj\" (UniqueName: \"kubernetes.io/projected/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-kube-api-access-jhwxj\") pod \"maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:13:56.851972 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.851863    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-maas-api-tls\") pod \"maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:13:56.851972 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.851908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-metrics-certs\") pod \"maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:13:56.851972 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.851956    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhwxj\" (UniqueName: \"kubernetes.io/projected/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-kube-api-access-jhwxj\") pod \"maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:13:56.852285 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.852047    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:13:56.852701 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.852677    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:13:56.854556 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.854534    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-maas-api-tls\") pod \"maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:13:56.854736 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.854715    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-metrics-certs\") pod \"maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:13:56.861061 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.861030    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhwxj\" (UniqueName: \"kubernetes.io/projected/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-kube-api-access-jhwxj\") pod \"maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:13:56.897928 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:56.897891    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:13:57.183651 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:57.176192    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"]
Sep 11 14:13:57.353853 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:57.353814    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j" event={"ID":"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc","Type":"ContainerStarted","Data":"17ea561509c289c6391d514ad0feec77da9472e7696806ff69ac29526d983992"}
Sep 11 14:13:58.360069 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:58.360028    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j" event={"ID":"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc","Type":"ContainerStarted","Data":"1ed32f9c9728911e5289434c3efa755ffdf15eb3f4c5b99853ec2222556fc17e"}
Sep 11 14:13:58.360490 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:58.360185    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:13:58.377079 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:13:58.377022    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j" podStartSLOduration=1.99680222 podStartE2EDuration="2.3770055s" totalImagesPullingTime="380.20328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:58.375981451 +0000 UTC m=+1519.773324202" watchObservedRunningTime="2026-09-11 14:13:58.3770055 +0000 UTC m=+1519.774348290"
Sep 11 14:14:04.370157 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:14:04.370070    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:15:12.266918 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.266860    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"]
Sep 11 14:15:12.273052 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.273020    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.275261 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.275230    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-24864d-openshift-default-dockercfg-qqbnx\""
Sep 11 14:15:12.285609 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.285571    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"]
Sep 11 14:15:12.359797 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.359752    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-workload-certs\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.359797 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.359796    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-token\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.360028 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.359813    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.360028 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.359834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-data\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.360028 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.359900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-875j7\" (UniqueName: \"kubernetes.io/projected/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-kube-api-access-875j7\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.360028 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.360022    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-workload-socket\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.360197 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.360071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-credential-socket\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.360197 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.360128    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-envoy\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.360197 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.360157    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-podinfo\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.461031 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.460983    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-workload-certs\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.461031 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.461028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-token\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.461031 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.461043    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.461366 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.461188    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-data\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.461366 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.461222    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-875j7\" (UniqueName: \"kubernetes.io/projected/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-kube-api-access-875j7\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.461366 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.461323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-workload-socket\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.461532 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.461396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-credential-socket\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.461532 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.461483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-envoy\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.461532 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.461513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-podinfo\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.461729 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.461578    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-workload-certs\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.461729 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.461611    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-workload-socket\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.461841 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.461784    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-data\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.461900 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.461840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.461900 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.461871    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-credential-socket\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.464107 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.464082    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-envoy\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.464336 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.464315    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-podinfo\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.469480 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.469445    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-token\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.469678 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.469457    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-875j7\" (UniqueName: \"kubernetes.io/projected/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-kube-api-access-875j7\") pod \"e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") " pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.586918 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.586874    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:12.862305 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.862232    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"]
Sep 11 14:15:12.864948 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:15:12.864914    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b6f4d67_2a12_4513_a7d7_db1110cbdd4f.slice/crio-a6c7ba4a555870ddf8bd2b214adaa304dd85e441ddce1991fa05677982c4b2d5 WatchSource:0}: Error finding container a6c7ba4a555870ddf8bd2b214adaa304dd85e441ddce1991fa05677982c4b2d5: Status 404 returned error can't find the container with id a6c7ba4a555870ddf8bd2b214adaa304dd85e441ddce1991fa05677982c4b2d5
Sep 11 14:15:12.867546 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.867510    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:15:12.867655 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.867581    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:15:12.867655 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:12.867611    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:15:13.726126 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:13.726085    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4" event={"ID":"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f","Type":"ContainerStarted","Data":"b11b79c66c623c78607c7f8bf02e5ae29fc21455cd8d730f356cbb69317cbf8b"}
Sep 11 14:15:13.726556 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:13.726137    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4" event={"ID":"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f","Type":"ContainerStarted","Data":"a6c7ba4a555870ddf8bd2b214adaa304dd85e441ddce1991fa05677982c4b2d5"}
Sep 11 14:15:13.748902 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:13.748830    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4" podStartSLOduration=1.748803935 podStartE2EDuration="1.748803935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:15:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:15:13.747221892 +0000 UTC m=+1595.144564638" watchObservedRunningTime="2026-09-11 14:15:13.748803935 +0000 UTC m=+1595.146146672"
Sep 11 14:15:14.587894 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:14.587847    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:14.593545 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:14.593500    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:14.731383 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:14.731353    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:14.732808 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:14.732787    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:16.819835 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:16.819792    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"]
Sep 11 14:15:16.823059 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:16.823029    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:16.825199 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:16.825168    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-24864d-serving-cert\""
Sep 11 14:15:16.835815 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:16.835783    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"]
Sep 11 14:15:16.905249 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:16.905204    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9015f5d-ebc7-41e5-9239-8d2da2322950-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:16.905455 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:16.905275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9015f5d-ebc7-41e5-9239-8d2da2322950-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:16.905529 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:16.905478    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9015f5d-ebc7-41e5-9239-8d2da2322950-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:16.905529 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:16.905512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4tks\" (UniqueName: \"kubernetes.io/projected/d9015f5d-ebc7-41e5-9239-8d2da2322950-kube-api-access-t4tks\") pod \"maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:17.006473 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:17.006431    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9015f5d-ebc7-41e5-9239-8d2da2322950-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:17.006473 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:17.006471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4tks\" (UniqueName: \"kubernetes.io/projected/d9015f5d-ebc7-41e5-9239-8d2da2322950-kube-api-access-t4tks\") pod \"maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:17.006820 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:17.006504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9015f5d-ebc7-41e5-9239-8d2da2322950-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:17.006820 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:17.006526    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9015f5d-ebc7-41e5-9239-8d2da2322950-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:17.006820 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:15:17.006677    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-serving-cert: secret "maas-api-e2e-ipp-cleanup-24864d-serving-cert" not found
Sep 11 14:15:17.006820 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:15:17.006768    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9015f5d-ebc7-41e5-9239-8d2da2322950-maas-api-tls podName:d9015f5d-ebc7-41e5-9239-8d2da2322950 nodeName:}" failed. No retries permitted until 2026-09-11 14:15:17.506743853 +0000 UTC m=+1598.904086567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d9015f5d-ebc7-41e5-9239-8d2da2322950-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm" (UID: "d9015f5d-ebc7-41e5-9239-8d2da2322950") : secret "maas-api-e2e-ipp-cleanup-24864d-serving-cert" not found
Sep 11 14:15:17.007279 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:17.007260    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9015f5d-ebc7-41e5-9239-8d2da2322950-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:17.009211 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:17.009191    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9015f5d-ebc7-41e5-9239-8d2da2322950-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:17.017428 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:17.017396    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4tks\" (UniqueName: \"kubernetes.io/projected/d9015f5d-ebc7-41e5-9239-8d2da2322950-kube-api-access-t4tks\") pod \"maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:17.511573 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:17.511534    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9015f5d-ebc7-41e5-9239-8d2da2322950-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:17.514200 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:17.514175    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9015f5d-ebc7-41e5-9239-8d2da2322950-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:17.736984 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:17.736939    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:18.013844 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:18.013811    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"]
Sep 11 14:15:18.016690 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:15:18.016659    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9015f5d_ebc7_41e5_9239_8d2da2322950.slice/crio-1aced851f95422bf47089a8af1cc21f2b6494052b591e3fa71c2c3819d7e2bff WatchSource:0}: Error finding container 1aced851f95422bf47089a8af1cc21f2b6494052b591e3fa71c2c3819d7e2bff: Status 404 returned error can't find the container with id 1aced851f95422bf47089a8af1cc21f2b6494052b591e3fa71c2c3819d7e2bff
Sep 11 14:15:18.750615 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:18.750510    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm" event={"ID":"d9015f5d-ebc7-41e5-9239-8d2da2322950","Type":"ContainerStarted","Data":"fa6459737bbc58f0c17815efc83e35f2be478cf7273bc056aa7d273b00c20777"}
Sep 11 14:15:18.750615 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:18.750576    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm" event={"ID":"d9015f5d-ebc7-41e5-9239-8d2da2322950","Type":"ContainerStarted","Data":"1aced851f95422bf47089a8af1cc21f2b6494052b591e3fa71c2c3819d7e2bff"}
Sep 11 14:15:18.750615 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:18.750606    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:18.767702 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:18.767615    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm" podStartSLOduration=2.298000433 podStartE2EDuration="2.767595042s" totalImagesPullingTime="469.594609ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:15:18.766217767 +0000 UTC m=+1600.163560519" watchObservedRunningTime="2026-09-11 14:15:18.767595042 +0000 UTC m=+1600.164937778"
Sep 11 14:15:24.762840 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:24.762804    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:35.841601 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:35.841564    2563 generic.go:356] "Generic (PLEG): container finished" podID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" containerID="b11b79c66c623c78607c7f8bf02e5ae29fc21455cd8d730f356cbb69317cbf8b" exitCode=0
Sep 11 14:15:35.842066 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:35.841615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4" event={"ID":"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f","Type":"ContainerDied","Data":"b11b79c66c623c78607c7f8bf02e5ae29fc21455cd8d730f356cbb69317cbf8b"}
Sep 11 14:15:35.842066 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:35.842059    2563 scope.go:117] "RemoveContainer" containerID="b11b79c66c623c78607c7f8bf02e5ae29fc21455cd8d730f356cbb69317cbf8b"
Sep 11 14:15:35.843241 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:35.843212    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:15:35.843304 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:35.843284    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:15:35.843344 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:35.843320    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 14:15:36.848892 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:36.848849    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4" event={"ID":"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f","Type":"ContainerStarted","Data":"ff245d4c497ecc901ce11ff0f9fc99d0b3b1166a45012fb76062d96e294016bc"}
Sep 11 14:15:37.587297 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:37.587258    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:37.587878 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:37.587827    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 11 14:15:37.588034 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:37.587944    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4" podUID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 11 14:15:38.588285 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:38.588245    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 11 14:15:38.589375 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:38.588904    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4" podUID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 11 14:15:39.588321 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:39.588279    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 11 14:15:39.588980 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:39.588945    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4" podUID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 11 14:15:40.587680 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:40.587612    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 11 14:15:40.587985 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:40.587959    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4" podUID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 11 14:15:41.587649 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:41.587581    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 11 14:15:41.588185 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:41.587697    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4" podUID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 11 14:15:42.592061 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:42.592029    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:42.592500 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:42.592303    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:42.593015 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:42.592992    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:47.505334 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.505300    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"]
Sep 11 14:15:47.505875 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.505555    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm" podUID="d9015f5d-ebc7-41e5-9239-8d2da2322950" containerName="maas-api" containerID="cri-o://fa6459737bbc58f0c17815efc83e35f2be478cf7273bc056aa7d273b00c20777" gracePeriod=30
Sep 11 14:15:47.766804 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.766735    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:47.903609 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.903566    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4tks\" (UniqueName: \"kubernetes.io/projected/d9015f5d-ebc7-41e5-9239-8d2da2322950-kube-api-access-t4tks\") pod \"d9015f5d-ebc7-41e5-9239-8d2da2322950\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") "
Sep 11 14:15:47.903815 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.903671    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9015f5d-ebc7-41e5-9239-8d2da2322950-openshift-service-ca\") pod \"d9015f5d-ebc7-41e5-9239-8d2da2322950\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") "
Sep 11 14:15:47.903815 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.903766    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9015f5d-ebc7-41e5-9239-8d2da2322950-maas-api-tls\") pod \"d9015f5d-ebc7-41e5-9239-8d2da2322950\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") "
Sep 11 14:15:47.903815 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.903793    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9015f5d-ebc7-41e5-9239-8d2da2322950-metrics-certs\") pod \"d9015f5d-ebc7-41e5-9239-8d2da2322950\" (UID: \"d9015f5d-ebc7-41e5-9239-8d2da2322950\") "
Sep 11 14:15:47.904059 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.904024    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9015f5d-ebc7-41e5-9239-8d2da2322950-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9015f5d-ebc7-41e5-9239-8d2da2322950" (UID: "d9015f5d-ebc7-41e5-9239-8d2da2322950"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:15:47.904147 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.904132    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9015f5d-ebc7-41e5-9239-8d2da2322950-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:15:47.906066 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.906023    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9015f5d-ebc7-41e5-9239-8d2da2322950-kube-api-access-t4tks" (OuterVolumeSpecName: "kube-api-access-t4tks") pod "d9015f5d-ebc7-41e5-9239-8d2da2322950" (UID: "d9015f5d-ebc7-41e5-9239-8d2da2322950"). InnerVolumeSpecName "kube-api-access-t4tks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:15:47.906218 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.906175    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9015f5d-ebc7-41e5-9239-8d2da2322950-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d9015f5d-ebc7-41e5-9239-8d2da2322950" (UID: "d9015f5d-ebc7-41e5-9239-8d2da2322950"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:15:47.906379 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.906356    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9015f5d-ebc7-41e5-9239-8d2da2322950-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9015f5d-ebc7-41e5-9239-8d2da2322950" (UID: "d9015f5d-ebc7-41e5-9239-8d2da2322950"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:15:47.918092 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.918060    2563 generic.go:356] "Generic (PLEG): container finished" podID="d9015f5d-ebc7-41e5-9239-8d2da2322950" containerID="fa6459737bbc58f0c17815efc83e35f2be478cf7273bc056aa7d273b00c20777" exitCode=0
Sep 11 14:15:47.918272 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.918120    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"
Sep 11 14:15:47.918272 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.918123    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm" event={"ID":"d9015f5d-ebc7-41e5-9239-8d2da2322950","Type":"ContainerDied","Data":"fa6459737bbc58f0c17815efc83e35f2be478cf7273bc056aa7d273b00c20777"}
Sep 11 14:15:47.918272 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.918240    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm" event={"ID":"d9015f5d-ebc7-41e5-9239-8d2da2322950","Type":"ContainerDied","Data":"1aced851f95422bf47089a8af1cc21f2b6494052b591e3fa71c2c3819d7e2bff"}
Sep 11 14:15:47.918272 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.918263    2563 scope.go:117] "RemoveContainer" containerID="fa6459737bbc58f0c17815efc83e35f2be478cf7273bc056aa7d273b00c20777"
Sep 11 14:15:47.934162 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.934126    2563 scope.go:117] "RemoveContainer" containerID="fa6459737bbc58f0c17815efc83e35f2be478cf7273bc056aa7d273b00c20777"
Sep 11 14:15:47.934738 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:15:47.934703    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa6459737bbc58f0c17815efc83e35f2be478cf7273bc056aa7d273b00c20777\": container with ID starting with fa6459737bbc58f0c17815efc83e35f2be478cf7273bc056aa7d273b00c20777 not found: ID does not exist" containerID="fa6459737bbc58f0c17815efc83e35f2be478cf7273bc056aa7d273b00c20777"
Sep 11 14:15:47.934879 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.934748    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa6459737bbc58f0c17815efc83e35f2be478cf7273bc056aa7d273b00c20777"} err="failed to get container status \"fa6459737bbc58f0c17815efc83e35f2be478cf7273bc056aa7d273b00c20777\": rpc error: code = NotFound desc = could not find container \"fa6459737bbc58f0c17815efc83e35f2be478cf7273bc056aa7d273b00c20777\": container with ID starting with fa6459737bbc58f0c17815efc83e35f2be478cf7273bc056aa7d273b00c20777 not found: ID does not exist"
Sep 11 14:15:47.944592 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.944557    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"]
Sep 11 14:15:47.949020 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:47.948988    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-24864d-7c75f9fbb7-nz9cm"]
Sep 11 14:15:48.005501 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:48.005464    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9015f5d-ebc7-41e5-9239-8d2da2322950-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:15:48.005501 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:48.005496    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9015f5d-ebc7-41e5-9239-8d2da2322950-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:15:48.005501 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:48.005506    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4tks\" (UniqueName: \"kubernetes.io/projected/d9015f5d-ebc7-41e5-9239-8d2da2322950-kube-api-access-t4tks\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:15:49.132496 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:49.132459    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9015f5d-ebc7-41e5-9239-8d2da2322950" path="/var/lib/kubelet/pods/d9015f5d-ebc7-41e5-9239-8d2da2322950/volumes"
Sep 11 14:15:54.229005 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:54.228964    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"]
Sep 11 14:15:54.229401 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:54.229244    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4" podUID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" containerName="istio-proxy" containerID="cri-o://ff245d4c497ecc901ce11ff0f9fc99d0b3b1166a45012fb76062d96e294016bc" gracePeriod=30
Sep 11 14:15:57.595368 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:57.595326    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 11 14:15:57.595800 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:57.595403    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4" podUID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 14:15:59.484457 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.484430    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:59.613344 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.613309    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-podinfo\") pod \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") "
Sep 11 14:15:59.613566 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.613353    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-875j7\" (UniqueName: \"kubernetes.io/projected/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-kube-api-access-875j7\") pod \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") "
Sep 11 14:15:59.613566 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.613374    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-envoy\") pod \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") "
Sep 11 14:15:59.613566 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.613433    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-data\") pod \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") "
Sep 11 14:15:59.613566 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.613452    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-workload-socket\") pod \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") "
Sep 11 14:15:59.613566 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.613475    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-credential-socket\") pod \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") "
Sep 11 14:15:59.613566 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.613538    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-workload-certs\") pod \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") "
Sep 11 14:15:59.613566 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.613560    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-token\") pod \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") "
Sep 11 14:15:59.613978 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.613584    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istiod-ca-cert\") pod \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\" (UID: \"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f\") "
Sep 11 14:15:59.613978 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.613818    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" (UID: "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:15:59.613978 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.613842    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" (UID: "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:15:59.613978 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.613836    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" (UID: "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:15:59.613978 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.613923    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-data\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:15:59.613978 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.613940    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-workload-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:15:59.614454 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.613956    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-credential-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:15:59.614454 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.614070    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" (UID: "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:15:59.614454 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.614107    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" (UID: "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:15:59.615983 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.615957    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" (UID: "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:15:59.616111 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.616002    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" (UID: "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:15:59.616234 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.616215    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" (UID: "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:15:59.616234 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.616225    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-kube-api-access-875j7" (OuterVolumeSpecName: "kube-api-access-875j7") pod "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" (UID: "3b6f4d67-2a12-4513-a7d7-db1110cbdd4f"). InnerVolumeSpecName "kube-api-access-875j7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:15:59.715048 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.715007    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-workload-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:15:59.715048 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.715041    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-token\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:15:59.715048 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.715049    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istiod-ca-cert\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:15:59.715048 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.715057    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-podinfo\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:15:59.715332 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.715066    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-875j7\" (UniqueName: \"kubernetes.io/projected/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-kube-api-access-875j7\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:15:59.715332 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.715075    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f-istio-envoy\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:15:59.981901 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.981808    2563 generic.go:356] "Generic (PLEG): container finished" podID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" containerID="ff245d4c497ecc901ce11ff0f9fc99d0b3b1166a45012fb76062d96e294016bc" exitCode=0
Sep 11 14:15:59.981901 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.981866    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"
Sep 11 14:15:59.982080 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.981894    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4" event={"ID":"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f","Type":"ContainerDied","Data":"ff245d4c497ecc901ce11ff0f9fc99d0b3b1166a45012fb76062d96e294016bc"}
Sep 11 14:15:59.982080 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.981955    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4" event={"ID":"3b6f4d67-2a12-4513-a7d7-db1110cbdd4f","Type":"ContainerDied","Data":"a6c7ba4a555870ddf8bd2b214adaa304dd85e441ddce1991fa05677982c4b2d5"}
Sep 11 14:15:59.982080 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.981980    2563 scope.go:117] "RemoveContainer" containerID="ff245d4c497ecc901ce11ff0f9fc99d0b3b1166a45012fb76062d96e294016bc"
Sep 11 14:15:59.991859 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:15:59.991840    2563 scope.go:117] "RemoveContainer" containerID="b11b79c66c623c78607c7f8bf02e5ae29fc21455cd8d730f356cbb69317cbf8b"
Sep 11 14:16:00.001327 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:00.001284    2563 scope.go:117] "RemoveContainer" containerID="ff245d4c497ecc901ce11ff0f9fc99d0b3b1166a45012fb76062d96e294016bc"
Sep 11 14:16:00.002182 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:16:00.002153    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff245d4c497ecc901ce11ff0f9fc99d0b3b1166a45012fb76062d96e294016bc\": container with ID starting with ff245d4c497ecc901ce11ff0f9fc99d0b3b1166a45012fb76062d96e294016bc not found: ID does not exist" containerID="ff245d4c497ecc901ce11ff0f9fc99d0b3b1166a45012fb76062d96e294016bc"
Sep 11 14:16:00.002309 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:00.002198    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff245d4c497ecc901ce11ff0f9fc99d0b3b1166a45012fb76062d96e294016bc"} err="failed to get container status \"ff245d4c497ecc901ce11ff0f9fc99d0b3b1166a45012fb76062d96e294016bc\": rpc error: code = NotFound desc = could not find container \"ff245d4c497ecc901ce11ff0f9fc99d0b3b1166a45012fb76062d96e294016bc\": container with ID starting with ff245d4c497ecc901ce11ff0f9fc99d0b3b1166a45012fb76062d96e294016bc not found: ID does not exist"
Sep 11 14:16:00.002309 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:00.002225    2563 scope.go:117] "RemoveContainer" containerID="b11b79c66c623c78607c7f8bf02e5ae29fc21455cd8d730f356cbb69317cbf8b"
Sep 11 14:16:00.002737 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:16:00.002703    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b11b79c66c623c78607c7f8bf02e5ae29fc21455cd8d730f356cbb69317cbf8b\": container with ID starting with b11b79c66c623c78607c7f8bf02e5ae29fc21455cd8d730f356cbb69317cbf8b not found: ID does not exist" containerID="b11b79c66c623c78607c7f8bf02e5ae29fc21455cd8d730f356cbb69317cbf8b"
Sep 11 14:16:00.002829 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:00.002741    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b11b79c66c623c78607c7f8bf02e5ae29fc21455cd8d730f356cbb69317cbf8b"} err="failed to get container status \"b11b79c66c623c78607c7f8bf02e5ae29fc21455cd8d730f356cbb69317cbf8b\": rpc error: code = NotFound desc = could not find container \"b11b79c66c623c78607c7f8bf02e5ae29fc21455cd8d730f356cbb69317cbf8b\": container with ID starting with b11b79c66c623c78607c7f8bf02e5ae29fc21455cd8d730f356cbb69317cbf8b not found: ID does not exist"
Sep 11 14:16:00.003687 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:00.003669    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"]
Sep 11 14:16:00.009320 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:00.009280    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-24864d-openshift-default-5dbc984b8d-lmvt4"]
Sep 11 14:16:01.131102 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:01.131068    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" path="/var/lib/kubelet/pods/3b6f4d67-2a12-4513-a7d7-db1110cbdd4f/volumes"
Sep 11 14:16:13.436478 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:13.436446    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"]
Sep 11 14:16:13.436951 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:13.436676    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw" podUID="aa54269d-cf0a-4259-a102-bd4a76557a4c" containerName="maas-api" containerID="cri-o://dda7c4920016710945137c2afffa086a5cd7bb462b5e3744ec5a772c79fae0d5" gracePeriod=30
Sep 11 14:16:13.688588 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:13.688517    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:16:13.742594 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:13.742556    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa54269d-cf0a-4259-a102-bd4a76557a4c-metrics-certs\") pod \"aa54269d-cf0a-4259-a102-bd4a76557a4c\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") "
Sep 11 14:16:13.742594 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:13.742594    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa54269d-cf0a-4259-a102-bd4a76557a4c-openshift-service-ca\") pod \"aa54269d-cf0a-4259-a102-bd4a76557a4c\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") "
Sep 11 14:16:13.742878 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:13.742652    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa54269d-cf0a-4259-a102-bd4a76557a4c-maas-api-tls\") pod \"aa54269d-cf0a-4259-a102-bd4a76557a4c\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") "
Sep 11 14:16:13.742878 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:13.742679    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sm56n\" (UniqueName: \"kubernetes.io/projected/aa54269d-cf0a-4259-a102-bd4a76557a4c-kube-api-access-sm56n\") pod \"aa54269d-cf0a-4259-a102-bd4a76557a4c\" (UID: \"aa54269d-cf0a-4259-a102-bd4a76557a4c\") "
Sep 11 14:16:13.743038 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:13.743009    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa54269d-cf0a-4259-a102-bd4a76557a4c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa54269d-cf0a-4259-a102-bd4a76557a4c" (UID: "aa54269d-cf0a-4259-a102-bd4a76557a4c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:16:13.744997 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:13.744954    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa54269d-cf0a-4259-a102-bd4a76557a4c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa54269d-cf0a-4259-a102-bd4a76557a4c" (UID: "aa54269d-cf0a-4259-a102-bd4a76557a4c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:16:13.745154 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:13.745085    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa54269d-cf0a-4259-a102-bd4a76557a4c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aa54269d-cf0a-4259-a102-bd4a76557a4c" (UID: "aa54269d-cf0a-4259-a102-bd4a76557a4c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:16:13.745220 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:13.745182    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa54269d-cf0a-4259-a102-bd4a76557a4c-kube-api-access-sm56n" (OuterVolumeSpecName: "kube-api-access-sm56n") pod "aa54269d-cf0a-4259-a102-bd4a76557a4c" (UID: "aa54269d-cf0a-4259-a102-bd4a76557a4c"). InnerVolumeSpecName "kube-api-access-sm56n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:16:13.843647 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:13.843581    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa54269d-cf0a-4259-a102-bd4a76557a4c-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:16:13.843647 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:13.843653    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa54269d-cf0a-4259-a102-bd4a76557a4c-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:16:13.843924 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:13.843669    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa54269d-cf0a-4259-a102-bd4a76557a4c-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:16:13.843924 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:13.843682    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sm56n\" (UniqueName: \"kubernetes.io/projected/aa54269d-cf0a-4259-a102-bd4a76557a4c-kube-api-access-sm56n\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:16:14.050123 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:14.050085    2563 generic.go:356] "Generic (PLEG): container finished" podID="aa54269d-cf0a-4259-a102-bd4a76557a4c" containerID="dda7c4920016710945137c2afffa086a5cd7bb462b5e3744ec5a772c79fae0d5" exitCode=0
Sep 11 14:16:14.050334 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:14.050140    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"
Sep 11 14:16:14.050334 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:14.050171    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw" event={"ID":"aa54269d-cf0a-4259-a102-bd4a76557a4c","Type":"ContainerDied","Data":"dda7c4920016710945137c2afffa086a5cd7bb462b5e3744ec5a772c79fae0d5"}
Sep 11 14:16:14.050334 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:14.050227    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw" event={"ID":"aa54269d-cf0a-4259-a102-bd4a76557a4c","Type":"ContainerDied","Data":"08287daf33725275e67c78c05960a5c5a61e1f231f6cd693ac32320b3111ea1c"}
Sep 11 14:16:14.050334 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:14.050250    2563 scope.go:117] "RemoveContainer" containerID="dda7c4920016710945137c2afffa086a5cd7bb462b5e3744ec5a772c79fae0d5"
Sep 11 14:16:14.059637 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:14.059606    2563 scope.go:117] "RemoveContainer" containerID="dda7c4920016710945137c2afffa086a5cd7bb462b5e3744ec5a772c79fae0d5"
Sep 11 14:16:14.060007 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:16:14.059986    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dda7c4920016710945137c2afffa086a5cd7bb462b5e3744ec5a772c79fae0d5\": container with ID starting with dda7c4920016710945137c2afffa086a5cd7bb462b5e3744ec5a772c79fae0d5 not found: ID does not exist" containerID="dda7c4920016710945137c2afffa086a5cd7bb462b5e3744ec5a772c79fae0d5"
Sep 11 14:16:14.060076 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:14.060015    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dda7c4920016710945137c2afffa086a5cd7bb462b5e3744ec5a772c79fae0d5"} err="failed to get container status \"dda7c4920016710945137c2afffa086a5cd7bb462b5e3744ec5a772c79fae0d5\": rpc error: code = NotFound desc = could not find container \"dda7c4920016710945137c2afffa086a5cd7bb462b5e3744ec5a772c79fae0d5\": container with ID starting with dda7c4920016710945137c2afffa086a5cd7bb462b5e3744ec5a772c79fae0d5 not found: ID does not exist"
Sep 11 14:16:14.070852 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:14.070820    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"]
Sep 11 14:16:14.075560 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:14.075519    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e3622e-76c7df679-cczqw"]
Sep 11 14:16:15.131099 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:16:15.131064    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa54269d-cf0a-4259-a102-bd4a76557a4c" path="/var/lib/kubelet/pods/aa54269d-cf0a-4259-a102-bd4a76557a4c/volumes"
Sep 11 14:17:02.780205 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:02.780113    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"]
Sep 11 14:17:02.780820 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:02.780437    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy" containerID="cri-o://4df63536b0dc9496f08dc243608be44db8f32e9be14cfa80644caffb2532862e" gracePeriod=30
Sep 11 14:17:06.341881 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:06.341832    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 14:17:06.342264 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:06.341925    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 14:17:08.026278 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.026250    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:17:08.143978 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.143938    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-token\") pod \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") "
Sep 11 14:17:08.143978 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.143997    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istiod-ca-cert\") pod \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") "
Sep 11 14:17:08.144264 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.144037    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-envoy\") pod \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") "
Sep 11 14:17:08.144264 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.144076    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-podinfo\") pod \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") "
Sep 11 14:17:08.144264 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.144101    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-workload-certs\") pod \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") "
Sep 11 14:17:08.144264 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.144133    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rbp5\" (UniqueName: \"kubernetes.io/projected/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-kube-api-access-7rbp5\") pod \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") "
Sep 11 14:17:08.144467 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.144324    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-workload-socket\") pod \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") "
Sep 11 14:17:08.144467 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.144386    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-data\") pod \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") "
Sep 11 14:17:08.144467 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.144415    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-credential-socket\") pod \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\" (UID: \"0aeb9f97-b9f8-4bf3-b26e-3c856609127b\") "
Sep 11 14:17:08.144467 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.144451    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0aeb9f97-b9f8-4bf3-b26e-3c856609127b" (UID: "0aeb9f97-b9f8-4bf3-b26e-3c856609127b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:17:08.144722 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.144466    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0aeb9f97-b9f8-4bf3-b26e-3c856609127b" (UID: "0aeb9f97-b9f8-4bf3-b26e-3c856609127b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:17:08.144780 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.144738    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-workload-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:08.144780 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.144760    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istiod-ca-cert\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:08.144780 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.144764    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-data" (OuterVolumeSpecName: "istio-data") pod "0aeb9f97-b9f8-4bf3-b26e-3c856609127b" (UID: "0aeb9f97-b9f8-4bf3-b26e-3c856609127b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:17:08.144905 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.144816    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0aeb9f97-b9f8-4bf3-b26e-3c856609127b" (UID: "0aeb9f97-b9f8-4bf3-b26e-3c856609127b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:17:08.144987 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.144965    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0aeb9f97-b9f8-4bf3-b26e-3c856609127b" (UID: "0aeb9f97-b9f8-4bf3-b26e-3c856609127b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:17:08.146459 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.146431    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-token" (OuterVolumeSpecName: "istio-token") pod "0aeb9f97-b9f8-4bf3-b26e-3c856609127b" (UID: "0aeb9f97-b9f8-4bf3-b26e-3c856609127b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:17:08.146598 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.146465    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0aeb9f97-b9f8-4bf3-b26e-3c856609127b" (UID: "0aeb9f97-b9f8-4bf3-b26e-3c856609127b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:17:08.146598 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.146508    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0aeb9f97-b9f8-4bf3-b26e-3c856609127b" (UID: "0aeb9f97-b9f8-4bf3-b26e-3c856609127b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 14:17:08.146855 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.146834    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-kube-api-access-7rbp5" (OuterVolumeSpecName: "kube-api-access-7rbp5") pod "0aeb9f97-b9f8-4bf3-b26e-3c856609127b" (UID: "0aeb9f97-b9f8-4bf3-b26e-3c856609127b"). InnerVolumeSpecName "kube-api-access-7rbp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:17:08.245543 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.245496    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-token\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:08.245543 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.245535    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-envoy\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:08.245543 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.245548    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-podinfo\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:08.245892 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.245562    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rbp5\" (UniqueName: \"kubernetes.io/projected/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-kube-api-access-7rbp5\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:08.245892 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.245576    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-workload-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:08.245892 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.245590    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-istio-data\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:08.245892 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.245603    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0aeb9f97-b9f8-4bf3-b26e-3c856609127b-credential-socket\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:08.322729 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.322689    2563 generic.go:356] "Generic (PLEG): container finished" podID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerID="4df63536b0dc9496f08dc243608be44db8f32e9be14cfa80644caffb2532862e" exitCode=0
Sep 11 14:17:08.322936 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.322747    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"
Sep 11 14:17:08.322936 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.322769    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" event={"ID":"0aeb9f97-b9f8-4bf3-b26e-3c856609127b","Type":"ContainerDied","Data":"4df63536b0dc9496f08dc243608be44db8f32e9be14cfa80644caffb2532862e"}
Sep 11 14:17:08.322936 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.322861    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg" event={"ID":"0aeb9f97-b9f8-4bf3-b26e-3c856609127b","Type":"ContainerDied","Data":"6f4616fcc28192dc6d4395289bfacddacb92b78e1ce509d6a54f0533b4d8816d"}
Sep 11 14:17:08.322936 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.322892    2563 scope.go:117] "RemoveContainer" containerID="4df63536b0dc9496f08dc243608be44db8f32e9be14cfa80644caffb2532862e"
Sep 11 14:17:08.332124 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.332100    2563 scope.go:117] "RemoveContainer" containerID="fd1c687e5d0ecec4f072394234701444e9ad789fe40381b60ee1d1bdbe8a7aad"
Sep 11 14:17:08.340978 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.340949    2563 scope.go:117] "RemoveContainer" containerID="4df63536b0dc9496f08dc243608be44db8f32e9be14cfa80644caffb2532862e"
Sep 11 14:17:08.341365 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:17:08.341339    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4df63536b0dc9496f08dc243608be44db8f32e9be14cfa80644caffb2532862e\": container with ID starting with 4df63536b0dc9496f08dc243608be44db8f32e9be14cfa80644caffb2532862e not found: ID does not exist" containerID="4df63536b0dc9496f08dc243608be44db8f32e9be14cfa80644caffb2532862e"
Sep 11 14:17:08.341442 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.341381    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4df63536b0dc9496f08dc243608be44db8f32e9be14cfa80644caffb2532862e"} err="failed to get container status \"4df63536b0dc9496f08dc243608be44db8f32e9be14cfa80644caffb2532862e\": rpc error: code = NotFound desc = could not find container \"4df63536b0dc9496f08dc243608be44db8f32e9be14cfa80644caffb2532862e\": container with ID starting with 4df63536b0dc9496f08dc243608be44db8f32e9be14cfa80644caffb2532862e not found: ID does not exist"
Sep 11 14:17:08.341442 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.341409    2563 scope.go:117] "RemoveContainer" containerID="fd1c687e5d0ecec4f072394234701444e9ad789fe40381b60ee1d1bdbe8a7aad"
Sep 11 14:17:08.341934 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:17:08.341896    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd1c687e5d0ecec4f072394234701444e9ad789fe40381b60ee1d1bdbe8a7aad\": container with ID starting with fd1c687e5d0ecec4f072394234701444e9ad789fe40381b60ee1d1bdbe8a7aad not found: ID does not exist" containerID="fd1c687e5d0ecec4f072394234701444e9ad789fe40381b60ee1d1bdbe8a7aad"
Sep 11 14:17:08.342100 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.341932    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd1c687e5d0ecec4f072394234701444e9ad789fe40381b60ee1d1bdbe8a7aad"} err="failed to get container status \"fd1c687e5d0ecec4f072394234701444e9ad789fe40381b60ee1d1bdbe8a7aad\": rpc error: code = NotFound desc = could not find container \"fd1c687e5d0ecec4f072394234701444e9ad789fe40381b60ee1d1bdbe8a7aad\": container with ID starting with fd1c687e5d0ecec4f072394234701444e9ad789fe40381b60ee1d1bdbe8a7aad not found: ID does not exist"
Sep 11 14:17:08.344037 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.344016    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"]
Sep 11 14:17:08.349807 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:08.349775    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-e3622e-openshift-default-7c876db644-prslg"]
Sep 11 14:17:09.131181 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:09.131146    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" path="/var/lib/kubelet/pods/0aeb9f97-b9f8-4bf3-b26e-3c856609127b/volumes"
Sep 11 14:17:13.852182 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:13.852139    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"]
Sep 11 14:17:13.852694 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:13.852441    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j" podUID="5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc" containerName="maas-api" containerID="cri-o://1ed32f9c9728911e5289434c3efa755ffdf15eb3f4c5b99853ec2222556fc17e" gracePeriod=30
Sep 11 14:17:14.126950 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.126914    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:17:14.199946 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.199905    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-metrics-certs\") pod \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") "
Sep 11 14:17:14.200129 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.199969    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-maas-api-tls\") pod \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") "
Sep 11 14:17:14.200129 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.200007    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhwxj\" (UniqueName: \"kubernetes.io/projected/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-kube-api-access-jhwxj\") pod \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") "
Sep 11 14:17:14.200129 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.200072    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-openshift-service-ca\") pod \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\" (UID: \"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc\") "
Sep 11 14:17:14.200827 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.200791    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc" (UID: "5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:17:14.202836 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.202786    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc" (UID: "5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:17:14.202836 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.202798    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc" (UID: "5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:17:14.203089 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.203060    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-kube-api-access-jhwxj" (OuterVolumeSpecName: "kube-api-access-jhwxj") pod "5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc" (UID: "5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc"). InnerVolumeSpecName "kube-api-access-jhwxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:17:14.300954 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.300917    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:14.300954 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.300950    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:14.300954 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.300959    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhwxj\" (UniqueName: \"kubernetes.io/projected/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-kube-api-access-jhwxj\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:14.301215 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.300968    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:14.353810 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.353769    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc" containerID="1ed32f9c9728911e5289434c3efa755ffdf15eb3f4c5b99853ec2222556fc17e" exitCode=0
Sep 11 14:17:14.353995 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.353848    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"
Sep 11 14:17:14.353995 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.353854    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j" event={"ID":"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc","Type":"ContainerDied","Data":"1ed32f9c9728911e5289434c3efa755ffdf15eb3f4c5b99853ec2222556fc17e"}
Sep 11 14:17:14.353995 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.353915    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j" event={"ID":"5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc","Type":"ContainerDied","Data":"17ea561509c289c6391d514ad0feec77da9472e7696806ff69ac29526d983992"}
Sep 11 14:17:14.353995 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.353937    2563 scope.go:117] "RemoveContainer" containerID="1ed32f9c9728911e5289434c3efa755ffdf15eb3f4c5b99853ec2222556fc17e"
Sep 11 14:17:14.363426 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.363404    2563 scope.go:117] "RemoveContainer" containerID="1ed32f9c9728911e5289434c3efa755ffdf15eb3f4c5b99853ec2222556fc17e"
Sep 11 14:17:14.363783 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:17:14.363762    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ed32f9c9728911e5289434c3efa755ffdf15eb3f4c5b99853ec2222556fc17e\": container with ID starting with 1ed32f9c9728911e5289434c3efa755ffdf15eb3f4c5b99853ec2222556fc17e not found: ID does not exist" containerID="1ed32f9c9728911e5289434c3efa755ffdf15eb3f4c5b99853ec2222556fc17e"
Sep 11 14:17:14.363855 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.363793    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed32f9c9728911e5289434c3efa755ffdf15eb3f4c5b99853ec2222556fc17e"} err="failed to get container status \"1ed32f9c9728911e5289434c3efa755ffdf15eb3f4c5b99853ec2222556fc17e\": rpc error: code = NotFound desc = could not find container \"1ed32f9c9728911e5289434c3efa755ffdf15eb3f4c5b99853ec2222556fc17e\": container with ID starting with 1ed32f9c9728911e5289434c3efa755ffdf15eb3f4c5b99853ec2222556fc17e not found: ID does not exist"
Sep 11 14:17:14.375476 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.375439    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"]
Sep 11 14:17:14.379382 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:14.379352    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-68d0c7-75c98d7cb5-cgm2j"]
Sep 11 14:17:15.131249 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:15.131216    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc" path="/var/lib/kubelet/pods/5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc/volumes"
Sep 11 14:17:31.426639 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:31.426583    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"]
Sep 11 14:17:31.427160 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:31.426828    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc" podUID="97297e22-0df7-4a1d-bc6d-c3255e415e6a" containerName="maas-api" containerID="cri-o://d9997cddbb1b1f52c6060fbdf28612d2809b23974bc0c3d325fd9f961df773c0" gracePeriod=30
Sep 11 14:17:31.680311 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:31.680244    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:17:31.750865 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:31.750829    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97297e22-0df7-4a1d-bc6d-c3255e415e6a-maas-api-tls\") pod \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") "
Sep 11 14:17:31.751061 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:31.750896    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97297e22-0df7-4a1d-bc6d-c3255e415e6a-metrics-certs\") pod \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") "
Sep 11 14:17:31.751061 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:31.750930    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njswn\" (UniqueName: \"kubernetes.io/projected/97297e22-0df7-4a1d-bc6d-c3255e415e6a-kube-api-access-njswn\") pod \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") "
Sep 11 14:17:31.751061 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:31.750991    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97297e22-0df7-4a1d-bc6d-c3255e415e6a-openshift-service-ca\") pod \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\" (UID: \"97297e22-0df7-4a1d-bc6d-c3255e415e6a\") "
Sep 11 14:17:31.751402 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:31.751369    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97297e22-0df7-4a1d-bc6d-c3255e415e6a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "97297e22-0df7-4a1d-bc6d-c3255e415e6a" (UID: "97297e22-0df7-4a1d-bc6d-c3255e415e6a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:17:31.753183 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:31.753148    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97297e22-0df7-4a1d-bc6d-c3255e415e6a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "97297e22-0df7-4a1d-bc6d-c3255e415e6a" (UID: "97297e22-0df7-4a1d-bc6d-c3255e415e6a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:17:31.753325 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:31.753202    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97297e22-0df7-4a1d-bc6d-c3255e415e6a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "97297e22-0df7-4a1d-bc6d-c3255e415e6a" (UID: "97297e22-0df7-4a1d-bc6d-c3255e415e6a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:17:31.753325 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:31.753243    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97297e22-0df7-4a1d-bc6d-c3255e415e6a-kube-api-access-njswn" (OuterVolumeSpecName: "kube-api-access-njswn") pod "97297e22-0df7-4a1d-bc6d-c3255e415e6a" (UID: "97297e22-0df7-4a1d-bc6d-c3255e415e6a"). InnerVolumeSpecName "kube-api-access-njswn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:17:31.852196 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:31.852148    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97297e22-0df7-4a1d-bc6d-c3255e415e6a-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:31.852196 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:31.852189    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97297e22-0df7-4a1d-bc6d-c3255e415e6a-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:31.852196 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:31.852199    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97297e22-0df7-4a1d-bc6d-c3255e415e6a-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:31.852196 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:31.852206    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njswn\" (UniqueName: \"kubernetes.io/projected/97297e22-0df7-4a1d-bc6d-c3255e415e6a-kube-api-access-njswn\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:32.441160 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:32.441125    2563 generic.go:356] "Generic (PLEG): container finished" podID="97297e22-0df7-4a1d-bc6d-c3255e415e6a" containerID="d9997cddbb1b1f52c6060fbdf28612d2809b23974bc0c3d325fd9f961df773c0" exitCode=0
Sep 11 14:17:32.441691 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:32.441176    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"
Sep 11 14:17:32.441691 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:32.441185    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc" event={"ID":"97297e22-0df7-4a1d-bc6d-c3255e415e6a","Type":"ContainerDied","Data":"d9997cddbb1b1f52c6060fbdf28612d2809b23974bc0c3d325fd9f961df773c0"}
Sep 11 14:17:32.441691 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:32.441242    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc" event={"ID":"97297e22-0df7-4a1d-bc6d-c3255e415e6a","Type":"ContainerDied","Data":"a8cb6704fdb6d1ba0d76fcc0b75358e584793fa60d3e2d9194675417dcfb5d8c"}
Sep 11 14:17:32.441691 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:32.441271    2563 scope.go:117] "RemoveContainer" containerID="d9997cddbb1b1f52c6060fbdf28612d2809b23974bc0c3d325fd9f961df773c0"
Sep 11 14:17:32.451158 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:32.451135    2563 scope.go:117] "RemoveContainer" containerID="d9997cddbb1b1f52c6060fbdf28612d2809b23974bc0c3d325fd9f961df773c0"
Sep 11 14:17:32.451505 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:17:32.451485    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9997cddbb1b1f52c6060fbdf28612d2809b23974bc0c3d325fd9f961df773c0\": container with ID starting with d9997cddbb1b1f52c6060fbdf28612d2809b23974bc0c3d325fd9f961df773c0 not found: ID does not exist" containerID="d9997cddbb1b1f52c6060fbdf28612d2809b23974bc0c3d325fd9f961df773c0"
Sep 11 14:17:32.451572 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:32.451516    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9997cddbb1b1f52c6060fbdf28612d2809b23974bc0c3d325fd9f961df773c0"} err="failed to get container status \"d9997cddbb1b1f52c6060fbdf28612d2809b23974bc0c3d325fd9f961df773c0\": rpc error: code = NotFound desc = could not find container \"d9997cddbb1b1f52c6060fbdf28612d2809b23974bc0c3d325fd9f961df773c0\": container with ID starting with d9997cddbb1b1f52c6060fbdf28612d2809b23974bc0c3d325fd9f961df773c0 not found: ID does not exist"
Sep 11 14:17:32.461545 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:32.461509    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"]
Sep 11 14:17:32.465791 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:32.465755    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-932c97-69cf7798b4-b2dxc"]
Sep 11 14:17:33.130973 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:33.130939    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97297e22-0df7-4a1d-bc6d-c3255e415e6a" path="/var/lib/kubelet/pods/97297e22-0df7-4a1d-bc6d-c3255e415e6a/volumes"
Sep 11 14:17:49.452136 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:49.452097    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"]
Sep 11 14:17:49.452620 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:49.452389    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq" podUID="be8da740-4454-471b-9b33-77afd2fc44e3" containerName="maas-api" containerID="cri-o://00201aad4047644a30d74cd734c829868c6529b2ca802b52d8e1d41514ab73dc" gracePeriod=30
Sep 11 14:17:49.705762 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:49.705691    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:17:49.708352 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:49.708325    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be8da740-4454-471b-9b33-77afd2fc44e3-openshift-service-ca\") pod \"be8da740-4454-471b-9b33-77afd2fc44e3\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") "
Sep 11 14:17:49.708502 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:49.708366    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-maas-api-tls\") pod \"be8da740-4454-471b-9b33-77afd2fc44e3\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") "
Sep 11 14:17:49.708502 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:49.708410    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfqh2\" (UniqueName: \"kubernetes.io/projected/be8da740-4454-471b-9b33-77afd2fc44e3-kube-api-access-wfqh2\") pod \"be8da740-4454-471b-9b33-77afd2fc44e3\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") "
Sep 11 14:17:49.708502 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:49.708442    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-metrics-certs\") pod \"be8da740-4454-471b-9b33-77afd2fc44e3\" (UID: \"be8da740-4454-471b-9b33-77afd2fc44e3\") "
Sep 11 14:17:49.708767 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:49.708738    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8da740-4454-471b-9b33-77afd2fc44e3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "be8da740-4454-471b-9b33-77afd2fc44e3" (UID: "be8da740-4454-471b-9b33-77afd2fc44e3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:17:49.710643 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:49.710605    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "be8da740-4454-471b-9b33-77afd2fc44e3" (UID: "be8da740-4454-471b-9b33-77afd2fc44e3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:17:49.710774 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:49.710742    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be8da740-4454-471b-9b33-77afd2fc44e3-kube-api-access-wfqh2" (OuterVolumeSpecName: "kube-api-access-wfqh2") pod "be8da740-4454-471b-9b33-77afd2fc44e3" (UID: "be8da740-4454-471b-9b33-77afd2fc44e3"). InnerVolumeSpecName "kube-api-access-wfqh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:17:49.710832 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:49.710809    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "be8da740-4454-471b-9b33-77afd2fc44e3" (UID: "be8da740-4454-471b-9b33-77afd2fc44e3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:17:49.809987 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:49.809950    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:49.809987 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:49.809990    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be8da740-4454-471b-9b33-77afd2fc44e3-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:49.810203 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:49.810005    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be8da740-4454-471b-9b33-77afd2fc44e3-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:49.810203 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:49.810016    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfqh2\" (UniqueName: \"kubernetes.io/projected/be8da740-4454-471b-9b33-77afd2fc44e3-kube-api-access-wfqh2\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:17:50.525502 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:50.525471    2563 generic.go:356] "Generic (PLEG): container finished" podID="be8da740-4454-471b-9b33-77afd2fc44e3" containerID="00201aad4047644a30d74cd734c829868c6529b2ca802b52d8e1d41514ab73dc" exitCode=0
Sep 11 14:17:50.526109 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:50.525514    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"
Sep 11 14:17:50.526109 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:50.525514    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq" event={"ID":"be8da740-4454-471b-9b33-77afd2fc44e3","Type":"ContainerDied","Data":"00201aad4047644a30d74cd734c829868c6529b2ca802b52d8e1d41514ab73dc"}
Sep 11 14:17:50.526109 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:50.525705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq" event={"ID":"be8da740-4454-471b-9b33-77afd2fc44e3","Type":"ContainerDied","Data":"78e04c6ba783f19a7522368c9ce8fcd2444a373b24fe5f11c095da1240f8155b"}
Sep 11 14:17:50.526109 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:50.525751    2563 scope.go:117] "RemoveContainer" containerID="00201aad4047644a30d74cd734c829868c6529b2ca802b52d8e1d41514ab73dc"
Sep 11 14:17:50.535119 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:50.535099    2563 scope.go:117] "RemoveContainer" containerID="00201aad4047644a30d74cd734c829868c6529b2ca802b52d8e1d41514ab73dc"
Sep 11 14:17:50.535421 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:17:50.535404    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00201aad4047644a30d74cd734c829868c6529b2ca802b52d8e1d41514ab73dc\": container with ID starting with 00201aad4047644a30d74cd734c829868c6529b2ca802b52d8e1d41514ab73dc not found: ID does not exist" containerID="00201aad4047644a30d74cd734c829868c6529b2ca802b52d8e1d41514ab73dc"
Sep 11 14:17:50.535459 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:50.535432    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00201aad4047644a30d74cd734c829868c6529b2ca802b52d8e1d41514ab73dc"} err="failed to get container status \"00201aad4047644a30d74cd734c829868c6529b2ca802b52d8e1d41514ab73dc\": rpc error: code = NotFound desc = could not find container \"00201aad4047644a30d74cd734c829868c6529b2ca802b52d8e1d41514ab73dc\": container with ID starting with 00201aad4047644a30d74cd734c829868c6529b2ca802b52d8e1d41514ab73dc not found: ID does not exist"
Sep 11 14:17:50.546963 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:50.546925    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"]
Sep 11 14:17:50.553576 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:50.553543    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-367517-6b8c6b8b5-wm5wq"]
Sep 11 14:17:51.131686 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:17:51.131648    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be8da740-4454-471b-9b33-77afd2fc44e3" path="/var/lib/kubelet/pods/be8da740-4454-471b-9b33-77afd2fc44e3/volumes"
Sep 11 14:20:38.702549 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:38.702508    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"]
Sep 11 14:20:38.705035 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:38.702767    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst" podUID="5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd" containerName="manager" containerID="cri-o://dd233e34b97db389bb978948aa5566186e517d30fa321c2d0d20f07529e7387d" gracePeriod=10
Sep 11 14:20:40.659163 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:40.659135    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"
Sep 11 14:20:40.733165 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:40.733078    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sc4h\" (UniqueName: \"kubernetes.io/projected/5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd-kube-api-access-8sc4h\") pod \"5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd\" (UID: \"5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd\") "
Sep 11 14:20:40.733165 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:40.733125    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd-extensions-socket-volume\") pod \"5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd\" (UID: \"5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd\") "
Sep 11 14:20:40.733546 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:40.733511    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd" (UID: "5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:20:40.735359 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:40.735327    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd-kube-api-access-8sc4h" (OuterVolumeSpecName: "kube-api-access-8sc4h") pod "5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd" (UID: "5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd"). InnerVolumeSpecName "kube-api-access-8sc4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:20:40.834547 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:40.834509    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sc4h\" (UniqueName: \"kubernetes.io/projected/5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd-kube-api-access-8sc4h\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:20:40.834547 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:40.834541    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd-extensions-socket-volume\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:20:41.298332 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:41.298301    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd" containerID="dd233e34b97db389bb978948aa5566186e517d30fa321c2d0d20f07529e7387d" exitCode=0
Sep 11 14:20:41.298516 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:41.298353    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"
Sep 11 14:20:41.298516 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:41.298390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst" event={"ID":"5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd","Type":"ContainerDied","Data":"dd233e34b97db389bb978948aa5566186e517d30fa321c2d0d20f07529e7387d"}
Sep 11 14:20:41.298516 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:41.298451    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst" event={"ID":"5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd","Type":"ContainerDied","Data":"4fd84e5813a4d0f34d7700b3cf7bb4eeb6d8755a30e4d0fb2f1c28dd6c3839af"}
Sep 11 14:20:41.298516 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:41.298478    2563 scope.go:117] "RemoveContainer" containerID="dd233e34b97db389bb978948aa5566186e517d30fa321c2d0d20f07529e7387d"
Sep 11 14:20:41.308001 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:41.307977    2563 scope.go:117] "RemoveContainer" containerID="dd233e34b97db389bb978948aa5566186e517d30fa321c2d0d20f07529e7387d"
Sep 11 14:20:41.308354 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:20:41.308334    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd233e34b97db389bb978948aa5566186e517d30fa321c2d0d20f07529e7387d\": container with ID starting with dd233e34b97db389bb978948aa5566186e517d30fa321c2d0d20f07529e7387d not found: ID does not exist" containerID="dd233e34b97db389bb978948aa5566186e517d30fa321c2d0d20f07529e7387d"
Sep 11 14:20:41.308394 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:41.308367    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd233e34b97db389bb978948aa5566186e517d30fa321c2d0d20f07529e7387d"} err="failed to get container status \"dd233e34b97db389bb978948aa5566186e517d30fa321c2d0d20f07529e7387d\": rpc error: code = NotFound desc = could not find container \"dd233e34b97db389bb978948aa5566186e517d30fa321c2d0d20f07529e7387d\": container with ID starting with dd233e34b97db389bb978948aa5566186e517d30fa321c2d0d20f07529e7387d not found: ID does not exist"
Sep 11 14:20:41.316679 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:41.316618    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"]
Sep 11 14:20:41.323132 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:41.323097    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4cfst"]
Sep 11 14:20:43.130355 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:20:43.130308    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd" path="/var/lib/kubelet/pods/5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd/volumes"
Sep 11 14:21:39.457914 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.457881    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs"]
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458198    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd" containerName="manager"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458209    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd" containerName="manager"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458218    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa54269d-cf0a-4259-a102-bd4a76557a4c" containerName="maas-api"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458224    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa54269d-cf0a-4259-a102-bd4a76557a4c" containerName="maas-api"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458233    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458238    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458244    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9015f5d-ebc7-41e5-9239-8d2da2322950" containerName="maas-api"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458249    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9015f5d-ebc7-41e5-9239-8d2da2322950" containerName="maas-api"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458259    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc" containerName="maas-api"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458264    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc" containerName="maas-api"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458270    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8da740-4454-471b-9b33-77afd2fc44e3" containerName="maas-api"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458275    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8da740-4454-471b-9b33-77afd2fc44e3" containerName="maas-api"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458282    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" containerName="istio-proxy"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458287    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" containerName="istio-proxy"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458294    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458301    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy"
Sep 11 14:21:39.458298 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458307    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97297e22-0df7-4a1d-bc6d-c3255e415e6a" containerName="maas-api"
Sep 11 14:21:39.458829 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458313    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="97297e22-0df7-4a1d-bc6d-c3255e415e6a" containerName="maas-api"
Sep 11 14:21:39.458829 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458381    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d1fb34a-8f9a-4ad2-8bf0-274a4d64c8fd" containerName="manager"
Sep 11 14:21:39.458829 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458389    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy"
Sep 11 14:21:39.458829 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458395    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="97297e22-0df7-4a1d-bc6d-c3255e415e6a" containerName="maas-api"
Sep 11 14:21:39.458829 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458400    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9015f5d-ebc7-41e5-9239-8d2da2322950" containerName="maas-api"
Sep 11 14:21:39.458829 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458406    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aeb9f97-b9f8-4bf3-b26e-3c856609127b" containerName="istio-proxy"
Sep 11 14:21:39.458829 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458411    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d66bf0f-5107-4c9a-8e54-9eab19e0c5bc" containerName="maas-api"
Sep 11 14:21:39.458829 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458418    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa54269d-cf0a-4259-a102-bd4a76557a4c" containerName="maas-api"
Sep 11 14:21:39.458829 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458424    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8da740-4454-471b-9b33-77afd2fc44e3" containerName="maas-api"
Sep 11 14:21:39.458829 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458429    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" containerName="istio-proxy"
Sep 11 14:21:39.458829 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.458434    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" containerName="istio-proxy"
Sep 11 14:21:39.462558 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.462534    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs"
Sep 11 14:21:39.465414 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.465387    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-882pd\""
Sep 11 14:21:39.476094 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.476062    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs"]
Sep 11 14:21:39.652508 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.652406    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnw8n\" (UniqueName: \"kubernetes.io/projected/90e55a7d-3867-481e-b33b-a0994475be8a-kube-api-access-hnw8n\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nkccs\" (UID: \"90e55a7d-3867-481e-b33b-a0994475be8a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs"
Sep 11 14:21:39.652508 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.652447    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90e55a7d-3867-481e-b33b-a0994475be8a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nkccs\" (UID: \"90e55a7d-3867-481e-b33b-a0994475be8a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs"
Sep 11 14:21:39.753865 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.753821    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnw8n\" (UniqueName: \"kubernetes.io/projected/90e55a7d-3867-481e-b33b-a0994475be8a-kube-api-access-hnw8n\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nkccs\" (UID: \"90e55a7d-3867-481e-b33b-a0994475be8a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs"
Sep 11 14:21:39.753865 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.753865    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90e55a7d-3867-481e-b33b-a0994475be8a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nkccs\" (UID: \"90e55a7d-3867-481e-b33b-a0994475be8a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs"
Sep 11 14:21:39.754311 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.754292    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90e55a7d-3867-481e-b33b-a0994475be8a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nkccs\" (UID: \"90e55a7d-3867-481e-b33b-a0994475be8a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs"
Sep 11 14:21:39.776422 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:39.776389    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnw8n\" (UniqueName: \"kubernetes.io/projected/90e55a7d-3867-481e-b33b-a0994475be8a-kube-api-access-hnw8n\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nkccs\" (UID: \"90e55a7d-3867-481e-b33b-a0994475be8a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs"
Sep 11 14:21:40.073297 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:40.073255    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs"
Sep 11 14:21:40.396311 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:40.396223    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs"]
Sep 11 14:21:40.399776 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:21:40.399745    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90e55a7d_3867_481e_b33b_a0994475be8a.slice/crio-c6742e0a1fd8838f4467f5e08b2417bd871c34b85666f2f84bc2c58343e087c8 WatchSource:0}: Error finding container c6742e0a1fd8838f4467f5e08b2417bd871c34b85666f2f84bc2c58343e087c8: Status 404 returned error can't find the container with id c6742e0a1fd8838f4467f5e08b2417bd871c34b85666f2f84bc2c58343e087c8
Sep 11 14:21:40.563176 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:40.563140    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs" event={"ID":"90e55a7d-3867-481e-b33b-a0994475be8a","Type":"ContainerStarted","Data":"1f528f3dabd3719955fb8e1c2c5b3be67a77cc3c8ad759e90ca0b2ae6e856cdf"}
Sep 11 14:21:40.563573 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:40.563226    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs" event={"ID":"90e55a7d-3867-481e-b33b-a0994475be8a","Type":"ContainerStarted","Data":"c6742e0a1fd8838f4467f5e08b2417bd871c34b85666f2f84bc2c58343e087c8"}
Sep 11 14:21:40.563573 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:40.563280    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs"
Sep 11 14:21:40.595427 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:40.595375    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs" podStartSLOduration=1.5953581 podStartE2EDuration="1.5953581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:21:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:21:40.595143151 +0000 UTC m=+1981.992485889" watchObservedRunningTime="2026-09-11 14:21:40.5953581 +0000 UTC m=+1981.992700837"
Sep 11 14:21:51.570498 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:21:51.570458    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nkccs"
Sep 11 14:23:13.159320 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.159283    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"]
Sep 11 14:23:13.159872 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.159655    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" containerName="istio-proxy"
Sep 11 14:23:13.159872 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.159671    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b6f4d67-2a12-4513-a7d7-db1110cbdd4f" containerName="istio-proxy"
Sep 11 14:23:13.162571 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.162551    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:13.173242 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.173208    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"]
Sep 11 14:23:13.289023 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.288972    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/916a6709-5cc9-4322-84ed-e2008fac5fcc-maas-api-tls\") pod \"maas-api-76677d8b8b-fqdlh\" (UID: \"916a6709-5cc9-4322-84ed-e2008fac5fcc\") " pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:13.289241 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.289039    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/916a6709-5cc9-4322-84ed-e2008fac5fcc-metrics-certs\") pod \"maas-api-76677d8b8b-fqdlh\" (UID: \"916a6709-5cc9-4322-84ed-e2008fac5fcc\") " pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:13.289241 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.289139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/916a6709-5cc9-4322-84ed-e2008fac5fcc-openshift-service-ca\") pod \"maas-api-76677d8b8b-fqdlh\" (UID: \"916a6709-5cc9-4322-84ed-e2008fac5fcc\") " pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:13.289241 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.289228    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmkfp\" (UniqueName: \"kubernetes.io/projected/916a6709-5cc9-4322-84ed-e2008fac5fcc-kube-api-access-bmkfp\") pod \"maas-api-76677d8b8b-fqdlh\" (UID: \"916a6709-5cc9-4322-84ed-e2008fac5fcc\") " pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:13.390525 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.390482    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmkfp\" (UniqueName: \"kubernetes.io/projected/916a6709-5cc9-4322-84ed-e2008fac5fcc-kube-api-access-bmkfp\") pod \"maas-api-76677d8b8b-fqdlh\" (UID: \"916a6709-5cc9-4322-84ed-e2008fac5fcc\") " pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:13.390728 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.390551    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/916a6709-5cc9-4322-84ed-e2008fac5fcc-maas-api-tls\") pod \"maas-api-76677d8b8b-fqdlh\" (UID: \"916a6709-5cc9-4322-84ed-e2008fac5fcc\") " pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:13.390728 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.390574    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/916a6709-5cc9-4322-84ed-e2008fac5fcc-metrics-certs\") pod \"maas-api-76677d8b8b-fqdlh\" (UID: \"916a6709-5cc9-4322-84ed-e2008fac5fcc\") " pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:13.390728 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.390595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/916a6709-5cc9-4322-84ed-e2008fac5fcc-openshift-service-ca\") pod \"maas-api-76677d8b8b-fqdlh\" (UID: \"916a6709-5cc9-4322-84ed-e2008fac5fcc\") " pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:13.391505 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.391451    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/916a6709-5cc9-4322-84ed-e2008fac5fcc-openshift-service-ca\") pod \"maas-api-76677d8b8b-fqdlh\" (UID: \"916a6709-5cc9-4322-84ed-e2008fac5fcc\") " pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:13.393363 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.393335    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/916a6709-5cc9-4322-84ed-e2008fac5fcc-maas-api-tls\") pod \"maas-api-76677d8b8b-fqdlh\" (UID: \"916a6709-5cc9-4322-84ed-e2008fac5fcc\") " pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:13.393490 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.393334    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/916a6709-5cc9-4322-84ed-e2008fac5fcc-metrics-certs\") pod \"maas-api-76677d8b8b-fqdlh\" (UID: \"916a6709-5cc9-4322-84ed-e2008fac5fcc\") " pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:13.399096 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.399063    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmkfp\" (UniqueName: \"kubernetes.io/projected/916a6709-5cc9-4322-84ed-e2008fac5fcc-kube-api-access-bmkfp\") pod \"maas-api-76677d8b8b-fqdlh\" (UID: \"916a6709-5cc9-4322-84ed-e2008fac5fcc\") " pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:13.475107 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.474999    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:13.757689 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.757662    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"]
Sep 11 14:23:13.760686 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:23:13.760654    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod916a6709_5cc9_4322_84ed_e2008fac5fcc.slice/crio-f06e8850d6134f7c919495b051ef6b5537ccf714011677add5fb638c128c504b WatchSource:0}: Error finding container f06e8850d6134f7c919495b051ef6b5537ccf714011677add5fb638c128c504b: Status 404 returned error can't find the container with id f06e8850d6134f7c919495b051ef6b5537ccf714011677add5fb638c128c504b
Sep 11 14:23:13.762514 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.762491    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:23:13.989923 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:13.989887    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh" event={"ID":"916a6709-5cc9-4322-84ed-e2008fac5fcc","Type":"ContainerStarted","Data":"f06e8850d6134f7c919495b051ef6b5537ccf714011677add5fb638c128c504b"}
Sep 11 14:23:14.996593 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:14.996557    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh" event={"ID":"916a6709-5cc9-4322-84ed-e2008fac5fcc","Type":"ContainerStarted","Data":"45cd62d55a69e80d80a9a65d64534cb861ee491d9e9312f9302fafe7a25156f5"}
Sep 11 14:23:14.997021 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:14.996727    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:15.019463 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:15.019396    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh" podStartSLOduration=1.506318318 podStartE2EDuration="2.019377157s" totalImagesPullingTime="513.058839ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:23:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:23:15.01824506 +0000 UTC m=+2076.415587797" watchObservedRunningTime="2026-09-11 14:23:15.019377157 +0000 UTC m=+2076.416719894"
Sep 11 14:23:21.007173 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:21.007136    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-76677d8b8b-fqdlh"
Sep 11 14:23:28.301833 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:28.301795    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-987c8478-jx8s2"]
Sep 11 14:23:28.302254 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:28.302060    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2" podUID="195f9393-d4e8-46a6-877f-ea37cbd766c2" containerName="maas-api" containerID="cri-o://15bc746b95f3c94c1ad275265bdb1d5021a4cf0d2f3191a4257b1953a4f67e5b" gracePeriod=30
Sep 11 14:23:29.394605 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:29.394557    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2" podUID="195f9393-d4e8-46a6-877f-ea37cbd766c2" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.26:8443/health\": dial tcp 10.133.0.26:8443: connect: connection refused"
Sep 11 14:23:29.547570 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:29.547530    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 14:23:29.640648 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:29.640528    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kswb\" (UniqueName: \"kubernetes.io/projected/195f9393-d4e8-46a6-877f-ea37cbd766c2-kube-api-access-6kswb\") pod \"195f9393-d4e8-46a6-877f-ea37cbd766c2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") "
Sep 11 14:23:29.640648 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:29.640568    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/195f9393-d4e8-46a6-877f-ea37cbd766c2-metrics-certs\") pod \"195f9393-d4e8-46a6-877f-ea37cbd766c2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") "
Sep 11 14:23:29.640866 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:29.640659    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/195f9393-d4e8-46a6-877f-ea37cbd766c2-maas-api-tls\") pod \"195f9393-d4e8-46a6-877f-ea37cbd766c2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") "
Sep 11 14:23:29.640866 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:29.640697    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/195f9393-d4e8-46a6-877f-ea37cbd766c2-openshift-service-ca\") pod \"195f9393-d4e8-46a6-877f-ea37cbd766c2\" (UID: \"195f9393-d4e8-46a6-877f-ea37cbd766c2\") "
Sep 11 14:23:29.641073 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:29.641049    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/195f9393-d4e8-46a6-877f-ea37cbd766c2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "195f9393-d4e8-46a6-877f-ea37cbd766c2" (UID: "195f9393-d4e8-46a6-877f-ea37cbd766c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:23:29.642875 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:29.642846    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/195f9393-d4e8-46a6-877f-ea37cbd766c2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "195f9393-d4e8-46a6-877f-ea37cbd766c2" (UID: "195f9393-d4e8-46a6-877f-ea37cbd766c2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:23:29.642991 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:29.642895    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/195f9393-d4e8-46a6-877f-ea37cbd766c2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "195f9393-d4e8-46a6-877f-ea37cbd766c2" (UID: "195f9393-d4e8-46a6-877f-ea37cbd766c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:23:29.642991 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:29.642898    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/195f9393-d4e8-46a6-877f-ea37cbd766c2-kube-api-access-6kswb" (OuterVolumeSpecName: "kube-api-access-6kswb") pod "195f9393-d4e8-46a6-877f-ea37cbd766c2" (UID: "195f9393-d4e8-46a6-877f-ea37cbd766c2"). InnerVolumeSpecName "kube-api-access-6kswb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:23:29.741602 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:29.741549    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/195f9393-d4e8-46a6-877f-ea37cbd766c2-maas-api-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:23:29.741602 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:29.741595    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/195f9393-d4e8-46a6-877f-ea37cbd766c2-openshift-service-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:23:29.741602 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:29.741608    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kswb\" (UniqueName: \"kubernetes.io/projected/195f9393-d4e8-46a6-877f-ea37cbd766c2-kube-api-access-6kswb\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:23:29.741602 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:29.741618    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/195f9393-d4e8-46a6-877f-ea37cbd766c2-metrics-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\""
Sep 11 14:23:30.065952 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:30.065917    2563 generic.go:356] "Generic (PLEG): container finished" podID="195f9393-d4e8-46a6-877f-ea37cbd766c2" containerID="15bc746b95f3c94c1ad275265bdb1d5021a4cf0d2f3191a4257b1953a4f67e5b" exitCode=0
Sep 11 14:23:30.066171 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:30.065968    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2"
Sep 11 14:23:30.066171 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:30.066013    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2" event={"ID":"195f9393-d4e8-46a6-877f-ea37cbd766c2","Type":"ContainerDied","Data":"15bc746b95f3c94c1ad275265bdb1d5021a4cf0d2f3191a4257b1953a4f67e5b"}
Sep 11 14:23:30.066171 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:30.066074    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-987c8478-jx8s2" event={"ID":"195f9393-d4e8-46a6-877f-ea37cbd766c2","Type":"ContainerDied","Data":"4546ca4d1de1d3a0603feb9b2c97b3671edaf50c688208303e06174f6384745b"}
Sep 11 14:23:30.066171 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:30.066105    2563 scope.go:117] "RemoveContainer" containerID="15bc746b95f3c94c1ad275265bdb1d5021a4cf0d2f3191a4257b1953a4f67e5b"
Sep 11 14:23:30.075557 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:30.075537    2563 scope.go:117] "RemoveContainer" containerID="15bc746b95f3c94c1ad275265bdb1d5021a4cf0d2f3191a4257b1953a4f67e5b"
Sep 11 14:23:30.075951 ip-10-0-128-161 kubenswrapper[2563]: E0911 14:23:30.075934    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15bc746b95f3c94c1ad275265bdb1d5021a4cf0d2f3191a4257b1953a4f67e5b\": container with ID starting with 15bc746b95f3c94c1ad275265bdb1d5021a4cf0d2f3191a4257b1953a4f67e5b not found: ID does not exist" containerID="15bc746b95f3c94c1ad275265bdb1d5021a4cf0d2f3191a4257b1953a4f67e5b"
Sep 11 14:23:30.076032 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:30.075959    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15bc746b95f3c94c1ad275265bdb1d5021a4cf0d2f3191a4257b1953a4f67e5b"} err="failed to get container status \"15bc746b95f3c94c1ad275265bdb1d5021a4cf0d2f3191a4257b1953a4f67e5b\": rpc error: code = NotFound desc = could not find container \"15bc746b95f3c94c1ad275265bdb1d5021a4cf0d2f3191a4257b1953a4f67e5b\": container with ID starting with 15bc746b95f3c94c1ad275265bdb1d5021a4cf0d2f3191a4257b1953a4f67e5b not found: ID does not exist"
Sep 11 14:23:30.088364 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:30.088326    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-987c8478-jx8s2"]
Sep 11 14:23:30.093765 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:30.093729    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-987c8478-jx8s2"]
Sep 11 14:23:31.130503 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:31.130467    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="195f9393-d4e8-46a6-877f-ea37cbd766c2" path="/var/lib/kubelet/pods/195f9393-d4e8-46a6-877f-ea37cbd766c2/volumes"
Sep 11 14:23:49.487659 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:49.487609    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-6ljtf_8e965efb-8de7-493e-9d18-4eb41c1250b2/manager/0.log"
Sep 11 14:23:50.336678 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:50.336640    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85cf544d9c-2x8r7_513da08b-e7f7-4ad2-bffd-a0971bce095a/copy-manifests/0.log"
Sep 11 14:23:50.353570 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:50.353535    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85cf544d9c-2x8r7_513da08b-e7f7-4ad2-bffd-a0971bce095a/manager/0.log"
Sep 11 14:23:50.966326 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:50.966291    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-76677d8b8b-fqdlh_916a6709-5cc9-4322-84ed-e2008fac5fcc/maas-api/0.log"
Sep 11 14:23:52.438062 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:52.438031    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-hxdgt_bd4dcebc-05a5-4526-a839-f76766b88025/manager/0.log"
Sep 11 14:23:52.687268 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:52.685424    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-nkccs_90e55a7d-3867-481e-b33b-a0994475be8a/manager/0.log"
Sep 11 14:23:52.959491 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:52.959462    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-frz57_7237fe7b-716c-4de1-93eb-5cdfdd681b88/manager/0.log"
Sep 11 14:23:53.443987 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:53.443951    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-bvw27_02021ad7-a325-4bda-8928-642506308f2c/discovery/0.log"
Sep 11 14:23:53.693872 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:53.693839    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-846c6f95fb-zqnpr_1e0ee83b-71a7-47ae-b510-631402cc6799/kube-auth-proxy/0.log"
Sep 11 14:23:54.587819 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:54.587786    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-qjrww_f2ede573-a192-4a2e-a5e7-7d57080ff0a4/main/0.log"
Sep 11 14:23:54.840555 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:54.840468    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-nkhbn_c155709a-b1b8-4f40-b0f4-96a43a176163/main/0.log"
Sep 11 14:23:55.097449 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:23:55.097365    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffp2np_e119a128-3ef5-4c89-972a-a9c53279b31a/main/0.log"
Sep 11 14:24:04.122799 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:04.122763    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hfjt6_2c293f3c-09e5-4f86-991b-1613158f19ee/global-pull-secret-syncer/0.log"
Sep 11 14:24:04.381947 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:04.381841    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kdcfl_cb93b48e-5389-4693-af4a-52e1e2163832/konnectivity-agent/0.log"
Sep 11 14:24:04.493614 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:04.493580    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-161.ec2.internal_fbf860b5b0097ab5f709024d70f9c3d5/haproxy/0.log"
Sep 11 14:24:09.689086 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:09.689058    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-hxdgt_bd4dcebc-05a5-4526-a839-f76766b88025/manager/0.log"
Sep 11 14:24:09.794780 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:09.794738    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-nkccs_90e55a7d-3867-481e-b33b-a0994475be8a/manager/0.log"
Sep 11 14:24:09.942399 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:09.942288    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-frz57_7237fe7b-716c-4de1-93eb-5cdfdd681b88/manager/0.log"
Sep 11 14:24:12.658706 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:12.658677    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-th29l_61776a5c-0574-4877-af8e-f94a5309536f/node-exporter/0.log"
Sep 11 14:24:12.685306 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:12.685275    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-th29l_61776a5c-0574-4877-af8e-f94a5309536f/kube-rbac-proxy/0.log"
Sep 11 14:24:12.713838 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:12.713812    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-th29l_61776a5c-0574-4877-af8e-f94a5309536f/init-textfile/0.log"
Sep 11 14:24:13.939647 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:13.939597    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"]
Sep 11 14:24:13.940041 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:13.939946    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="195f9393-d4e8-46a6-877f-ea37cbd766c2" containerName="maas-api"
Sep 11 14:24:13.940041 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:13.939957    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="195f9393-d4e8-46a6-877f-ea37cbd766c2" containerName="maas-api"
Sep 11 14:24:13.940041 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:13.940018    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="195f9393-d4e8-46a6-877f-ea37cbd766c2" containerName="maas-api"
Sep 11 14:24:13.942888 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:13.942861    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:13.945117 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:13.945096    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kdmhs\"/\"default-dockercfg-fw2z6\""
Sep 11 14:24:13.945510 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:13.945490    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kdmhs\"/\"openshift-service-ca.crt\""
Sep 11 14:24:13.945957 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:13.945930    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kdmhs\"/\"kube-root-ca.crt\""
Sep 11 14:24:13.963530 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:13.963491    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"]
Sep 11 14:24:14.018591 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.018554    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/788e8cb7-dc88-4d95-9ac5-c4a188423004-lib-modules\") pod \"perf-node-gather-daemonset-gmfqk\" (UID: \"788e8cb7-dc88-4d95-9ac5-c4a188423004\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.018591 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.018601    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/788e8cb7-dc88-4d95-9ac5-c4a188423004-proc\") pod \"perf-node-gather-daemonset-gmfqk\" (UID: \"788e8cb7-dc88-4d95-9ac5-c4a188423004\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.018847 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.018661    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/788e8cb7-dc88-4d95-9ac5-c4a188423004-podres\") pod \"perf-node-gather-daemonset-gmfqk\" (UID: \"788e8cb7-dc88-4d95-9ac5-c4a188423004\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.018847 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.018770    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8jw6\" (UniqueName: \"kubernetes.io/projected/788e8cb7-dc88-4d95-9ac5-c4a188423004-kube-api-access-v8jw6\") pod \"perf-node-gather-daemonset-gmfqk\" (UID: \"788e8cb7-dc88-4d95-9ac5-c4a188423004\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.018847 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.018831    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/788e8cb7-dc88-4d95-9ac5-c4a188423004-sys\") pod \"perf-node-gather-daemonset-gmfqk\" (UID: \"788e8cb7-dc88-4d95-9ac5-c4a188423004\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.119784 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.119740    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/788e8cb7-dc88-4d95-9ac5-c4a188423004-lib-modules\") pod \"perf-node-gather-daemonset-gmfqk\" (UID: \"788e8cb7-dc88-4d95-9ac5-c4a188423004\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.119784 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.119787    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/788e8cb7-dc88-4d95-9ac5-c4a188423004-proc\") pod \"perf-node-gather-daemonset-gmfqk\" (UID: \"788e8cb7-dc88-4d95-9ac5-c4a188423004\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.120020 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.119806    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/788e8cb7-dc88-4d95-9ac5-c4a188423004-podres\") pod \"perf-node-gather-daemonset-gmfqk\" (UID: \"788e8cb7-dc88-4d95-9ac5-c4a188423004\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.120020 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.119867    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8jw6\" (UniqueName: \"kubernetes.io/projected/788e8cb7-dc88-4d95-9ac5-c4a188423004-kube-api-access-v8jw6\") pod \"perf-node-gather-daemonset-gmfqk\" (UID: \"788e8cb7-dc88-4d95-9ac5-c4a188423004\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.120020 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.119905    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/788e8cb7-dc88-4d95-9ac5-c4a188423004-podres\") pod \"perf-node-gather-daemonset-gmfqk\" (UID: \"788e8cb7-dc88-4d95-9ac5-c4a188423004\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.120020 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.119876    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/788e8cb7-dc88-4d95-9ac5-c4a188423004-proc\") pod \"perf-node-gather-daemonset-gmfqk\" (UID: \"788e8cb7-dc88-4d95-9ac5-c4a188423004\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.120020 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.119919    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/788e8cb7-dc88-4d95-9ac5-c4a188423004-lib-modules\") pod \"perf-node-gather-daemonset-gmfqk\" (UID: \"788e8cb7-dc88-4d95-9ac5-c4a188423004\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.120020 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.119938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/788e8cb7-dc88-4d95-9ac5-c4a188423004-sys\") pod \"perf-node-gather-daemonset-gmfqk\" (UID: \"788e8cb7-dc88-4d95-9ac5-c4a188423004\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.120020 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.119996    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/788e8cb7-dc88-4d95-9ac5-c4a188423004-sys\") pod \"perf-node-gather-daemonset-gmfqk\" (UID: \"788e8cb7-dc88-4d95-9ac5-c4a188423004\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.129649 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.129587    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8jw6\" (UniqueName: \"kubernetes.io/projected/788e8cb7-dc88-4d95-9ac5-c4a188423004-kube-api-access-v8jw6\") pod \"perf-node-gather-daemonset-gmfqk\" (UID: \"788e8cb7-dc88-4d95-9ac5-c4a188423004\") " pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.254679 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.254567    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:14.528282 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:14.528253    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"]
Sep 11 14:24:14.530688 ip-10-0-128-161 kubenswrapper[2563]: W0911 14:24:14.530652    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod788e8cb7_dc88_4d95_9ac5_c4a188423004.slice/crio-18e61581def51e76e27221e3a2a42c75fda61d2d9d5e7ce134f35c75d70e61ed WatchSource:0}: Error finding container 18e61581def51e76e27221e3a2a42c75fda61d2d9d5e7ce134f35c75d70e61ed: Status 404 returned error can't find the container with id 18e61581def51e76e27221e3a2a42c75fda61d2d9d5e7ce134f35c75d70e61ed
Sep 11 14:24:15.268427 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:15.268386    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk" event={"ID":"788e8cb7-dc88-4d95-9ac5-c4a188423004","Type":"ContainerStarted","Data":"6164ebf3ccc3ab6066aedafff93b093d9243945e9a2f04b96f59de9e8bc659d4"}
Sep 11 14:24:15.268855 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:15.268447    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk" event={"ID":"788e8cb7-dc88-4d95-9ac5-c4a188423004","Type":"ContainerStarted","Data":"18e61581def51e76e27221e3a2a42c75fda61d2d9d5e7ce134f35c75d70e61ed"}
Sep 11 14:24:15.268855 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:15.268493    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:15.286717 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:15.286668    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk" podStartSLOduration=2.286649133 podStartE2EDuration="2.286649133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 14:24:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:24:15.285608094 +0000 UTC m=+2136.682950848" watchObservedRunningTime="2026-09-11 14:24:15.286649133 +0000 UTC m=+2136.683991864"
Sep 11 14:24:17.915787 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:17.915759    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rxc9q_2054806b-ed98-4a9c-b889-0a4efda17d75/dns/0.log"
Sep 11 14:24:17.950522 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:17.950493    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rxc9q_2054806b-ed98-4a9c-b889-0a4efda17d75/kube-rbac-proxy/0.log"
Sep 11 14:24:18.085433 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:18.085400    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7q9dt_071b3477-ebf8-450b-9ca7-c2767dea40ea/dns-node-resolver/0.log"
Sep 11 14:24:18.730673 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:18.730615    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7f77897478-vbwcm_6a67eff1-dc1f-49ed-b3c4-24ab86713afc/registry/0.log"
Sep 11 14:24:18.781787 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:18.781757    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5frp2_d39e8d10-0ae8-4265-9b99-ac2ac072b1ab/node-ca/0.log"
Sep 11 14:24:20.142133 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:20.142093    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-bvw27_02021ad7-a325-4bda-8928-642506308f2c/discovery/0.log"
Sep 11 14:24:20.198832 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:20.198797    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-846c6f95fb-zqnpr_1e0ee83b-71a7-47ae-b510-631402cc6799/kube-auth-proxy/0.log"
Sep 11 14:24:21.283815 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:21.283785    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kdmhs/perf-node-gather-daemonset-gmfqk"
Sep 11 14:24:21.331377 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:21.331346    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8cwcd_a3d7f55a-5d08-4e04-a09c-73c237ab55e8/serve-healthcheck-canary/0.log"
Sep 11 14:24:21.911991 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:21.911963    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pzd6j_82fbbb40-345c-41b3-ac15-49360dcd7859/kube-rbac-proxy/0.log"
Sep 11 14:24:21.945068 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:21.945036    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pzd6j_82fbbb40-345c-41b3-ac15-49360dcd7859/exporter/0.log"
Sep 11 14:24:21.985338 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:21.985306    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pzd6j_82fbbb40-345c-41b3-ac15-49360dcd7859/extractor/0.log"
Sep 11 14:24:24.743266 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:24.743185    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-6ljtf_8e965efb-8de7-493e-9d18-4eb41c1250b2/manager/0.log"
Sep 11 14:24:25.433821 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:25.433734    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-bvw27_02021ad7-a325-4bda-8928-642506308f2c/discovery/0.log"
Sep 11 14:24:25.485240 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:25.485202    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85cf544d9c-2x8r7_513da08b-e7f7-4ad2-bffd-a0971bce095a/manager/0.log"
Sep 11 14:24:25.489486 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:25.489440    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-846c6f95fb-zqnpr_1e0ee83b-71a7-47ae-b510-631402cc6799/kube-auth-proxy/0.log"
Sep 11 14:24:25.516703 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:25.516675    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85cf544d9c-2x8r7_513da08b-e7f7-4ad2-bffd-a0971bce095a/copy-manifests/0.log"
Sep 11 14:24:27.515000 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:27.514958    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-bvw27_02021ad7-a325-4bda-8928-642506308f2c/discovery/0.log"
Sep 11 14:24:27.573879 ip-10-0-128-161 kubenswrapper[2563]: I0911 14:24:27.573849    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-846c6f95fb-zqnpr_1e0ee83b-71a7-47ae-b510-631402cc6799/kube-auth-proxy/0.log"