Sep 11 12:55:44.900087 ip-10-0-143-6 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 12:55:45.264608 ip-10-0-143-6 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 12:55:45.264608 ip-10-0-143-6 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 12:55:45.264608 ip-10-0-143-6 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 12:55:45.264608 ip-10-0-143-6 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 12:55:45.265179 ip-10-0-143-6 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 12:55:45.267293 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.267225    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 12:55:45.270842 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270827    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 12:55:45.270842 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270841    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270844    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270848    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270852    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270855    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270858    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270861    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270864    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270867    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270870    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270872    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270875    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270877    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270880    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270882    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270886    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270889    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270891    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270894    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270896    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 12:55:45.270910 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270899    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270901    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270904    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270907    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270909    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270912    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270915    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270918    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270921    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270924    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270926    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270929    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270931    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270934    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270937    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270939    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270942    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270944    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270949    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270953    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 12:55:45.271417 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270956    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270959    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270961    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270964    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270966    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270969    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270972    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270974    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270977    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270979    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270983    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270986    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270989    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270992    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270995    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.270999    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271001    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271005    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271008    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 12:55:45.271909 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271011    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271013    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271016    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271019    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271021    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271025    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271027    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271030    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271033    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271035    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271039    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271043    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271046    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271050    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271053    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271055    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271058    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271061    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271064    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 12:55:45.272374 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271066    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271069    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271072    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271074    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271078    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271081    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271083    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271086    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271090    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271093    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271096    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271098    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271101    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271104    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271106    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271109    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271112    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271115    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271117    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.271120    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 12:55:45.272848 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272667    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272675    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272678    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272682    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272685    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272688    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272691    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272693    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272696    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272699    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272701    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272705    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272708    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272710    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272713    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272716    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272720    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272724    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272727    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272730    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 12:55:45.273343 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272734    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272736    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272739    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272742    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272744    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272747    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272750    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272752    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272755    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272757    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272760    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272762    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272765    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272767    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272770    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272773    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272776    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272779    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272785    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 12:55:45.273886 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272788    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272790    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272794    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272798    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272801    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272804    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272820    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272823    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272826    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272829    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272831    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272834    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272837    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272839    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272842    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272845    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272847    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272850    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272853    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272855    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 12:55:45.274355 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272858    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272862    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272865    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272868    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272871    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272874    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272877    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272879    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272882    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272885    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272888    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272890    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272894    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272896    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272899    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272902    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272905    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272908    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272910    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 12:55:45.274861 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272914    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272916    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272919    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272922    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272924    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272926    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272929    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272932    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272935    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272937    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272940    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272943    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272945    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272948    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272951    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272953    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272956    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272958    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272961    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272963    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 12:55:45.275331 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.272966    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273358    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273364    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273367    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273370    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273373    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273377    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273381    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273384    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273387    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273390    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273393    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273395    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273398    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273401    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273404    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273406    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273410    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273412    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273415    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 12:55:45.275819 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273417    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273420    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273423    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273425    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273428    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273431    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273433    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273436    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273439    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273442    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273444    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273447    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273449    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273452    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273454    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273457    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273459    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273462    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273465    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273467    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 12:55:45.276309 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273470    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273472    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273475    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273477    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273480    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273483    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273485    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273488    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273491    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273494    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273497    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273500    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273502    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273505    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273507    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273510    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273512    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273515    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273518    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 12:55:45.276792 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273521    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273524    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273526    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273529    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273531    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273534    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273536    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273539    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273541    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273543    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273546    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273548    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273556    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273559    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273562    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273564    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273567    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273569    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273572    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273574    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 12:55:45.277271 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273577    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273580    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273582    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273585    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273587    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273590    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273592    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273596    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273599    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273602    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273605    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273608    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273611    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273615    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273618    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273620    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273623    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273626    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273628    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 12:55:45.277744 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273631    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.273633    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273704    2582 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273748    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273754    2582 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273759    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273770    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273773    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273777    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273781    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273787    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273791    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273794    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273797    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273800    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273803    2582 flags.go:64] FLAG: --cgroup-root=""
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273821    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273825    2582 flags.go:64] FLAG: --client-ca-file=""
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273828    2582 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273831    2582 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273838    2582 flags.go:64] FLAG: --cluster-domain=""
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273841    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273844    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273848    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 12:55:45.278242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273852    2582 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273856    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273858    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273861    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273865    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273868    2582 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273871    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273874    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273877    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273879    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273883    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273887    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273889    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273892    2582 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273896    2582 flags.go:64] FLAG: --enable-server="true"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273904    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273911    2582 flags.go:64] FLAG: --event-burst="100"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273915    2582 flags.go:64] FLAG: --event-qps="50"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273919    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273922    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273925    2582 flags.go:64] FLAG: --eviction-hard=""
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273928    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273931    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273934    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273937    2582 flags.go:64] FLAG: --eviction-soft=""
Sep 11 12:55:45.278903 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273940    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273943    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273946    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273949    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273951    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273954    2582 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273958    2582 flags.go:64] FLAG: --feature-gates=""
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273961    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273964    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273967    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273970    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273974    2582 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273977    2582 flags.go:64] FLAG: --help="false"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273979    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273982    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273985    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273988    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273992    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273995    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.273998    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274001    2582 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274003    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274006    2582 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 12:55:45.279543 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274010    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274013    2582 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274018    2582 flags.go:64] FLAG: --kube-reserved=""
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274021    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274024    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274027    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274029    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274032    2582 flags.go:64] FLAG: --lock-file=""
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274044    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274048    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274051    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274056    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274059    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274062    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274065    2582 flags.go:64] FLAG: --logging-format="text"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274069    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274073    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274076    2582 flags.go:64] FLAG: --manifest-url=""
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274079    2582 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274083    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274086    2582 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274090    2582 flags.go:64] FLAG: --max-pods="110"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274093    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274096    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274099    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 12:55:45.280142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274101    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274104    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274107    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274110    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274117    2582 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274120    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274123    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274126    2582 flags.go:64] FLAG: --pod-cidr=""
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274130    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274135    2582 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274139    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274142    2582 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274145    2582 flags.go:64] FLAG: --port="10250"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274148    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274151    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d992f3c4bd7f50c7"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274154    2582 flags.go:64] FLAG: --qos-reserved=""
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274157    2582 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274160    2582 flags.go:64] FLAG: --register-node="true"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274163    2582 flags.go:64] FLAG: --register-with-taints=""
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274167    2582 flags.go:64] FLAG: --registry-burst="10"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274170    2582 flags.go:64] FLAG: --registry-qps="5"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274173    2582 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274177    2582 flags.go:64] FLAG: --reserved-memory=""
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274181    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274184    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 12:55:45.280757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274187    2582 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274190    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274193    2582 flags.go:64] FLAG: --runonce="false"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274196    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274199    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274202    2582 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274205    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274208    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274211    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274214    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274217    2582 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274220    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274223    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274226    2582 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274228    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274231    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274235    2582 flags.go:64] FLAG: --system-cgroups=""
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274239    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274244    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274247    2582 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274250    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274254    2582 flags.go:64] FLAG: --tls-min-version=""
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274257    2582 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274260    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274263    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 12:55:45.281385 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274266    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274269    2582 flags.go:64] FLAG: --v="2"
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274273    2582 flags.go:64] FLAG: --version="false"
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274277    2582 flags.go:64] FLAG: --vmodule=""
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274282    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.274285    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274394    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274398    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274402    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274405    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274408    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274411    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274414    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274418    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274422    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274425    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274428    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274431    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274433    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274437    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274439    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 12:55:45.282016 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274442    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274445    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274448    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274453    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274455    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274458    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274461    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274464    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274467    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274470    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274472    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274475    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274477    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274480    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274483    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274486    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274490    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274494    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274497    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274501    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 12:55:45.282528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274503    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274506    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274510    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274514    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274517    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274520    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274522    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274525    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274529    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274531    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274534    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274536    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274539    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274542    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274544    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274548    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274551    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274554    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274557    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274560    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 12:55:45.283063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274562    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274565    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274568    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274570    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274573    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274576    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274578    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274582    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274585    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274588    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274591    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274593    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274596    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274599    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274602    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274604    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274607    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274610    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274612    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274615    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 12:55:45.283548 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274618    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274620    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274623    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274626    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274628    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274631    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274634    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274638    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274641    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274644    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274647    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274649    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274652    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274654    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274657    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274660    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274662    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274665    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274668    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 12:55:45.284093 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274671    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 12:55:45.284562 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274674    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 12:55:45.284562 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274677    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 12:55:45.284562 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274680    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 12:55:45.284562 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.274682    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 12:55:45.284562 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.275323    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 12:55:45.284562 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.281945    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 12:55:45.284562 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.281965    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 12:55:45.284562 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.282729    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 12:55:45.284562 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.282738    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 12:55:45.284562 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282791    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 12:55:45.284562 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282796    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 12:55:45.284562 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282801    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 12:55:45.284562 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282805    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 12:55:45.284562 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282823    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282827    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282830    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282833    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282836    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282838    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282841    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282844    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282846    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282849    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282852    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282854    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282857    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282860    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282863    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282867    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282870    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282873    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282876    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 12:55:45.284917 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282879    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282881    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282884    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282887    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282889    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282892    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282896    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282900    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282902    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282905    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282908    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282910    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282913    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282916    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282918    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282921    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282923    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282926    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282928    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282931    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 12:55:45.285386 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282933    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282936    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282938    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282941    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282943    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282946    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282949    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282951    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282956    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282959    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282962    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282965    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282968    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282971    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282974    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282976    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282979    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282982    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282986    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 12:55:45.285894 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282989    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282992    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282995    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.282998    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283001    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283004    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283007    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283010    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283013    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283016    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283018    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283021    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283023    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283026    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283028    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283031    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283033    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283036    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283039    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283041    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 12:55:45.286398 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283044    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283046    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283049    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283052    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283054    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283057    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283060    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283062    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283065    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283067    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283070    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283073    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283076    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283079    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283082    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283085    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 12:55:45.286889 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283087    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 12:55:45.287266 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.283092    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 12:55:45.287266 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283242    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 12:55:45.287266 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283249    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 12:55:45.287266 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283253    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 12:55:45.287266 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283256    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 12:55:45.287266 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283259    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 12:55:45.287266 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283262    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 12:55:45.287266 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283265    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 12:55:45.287266 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283267    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 12:55:45.287266 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283270    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 12:55:45.287266 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283273    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 12:55:45.287266 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283276    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 12:55:45.287266 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283278    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 12:55:45.287266 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283281    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 12:55:45.287266 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283284    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283287    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283289    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283292    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283295    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283298    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283300    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283303    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283305    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283308    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283311    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283314    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283317    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283319    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283322    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283331    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283335    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283339    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283343    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 12:55:45.287627 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283346    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283350    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283353    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283355    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283358    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283361    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283363    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283366    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283369    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283371    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283374    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283376    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283379    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283382    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283384    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283387    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283389    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283392    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283395    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283397    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 12:55:45.288165 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283400    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283402    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283405    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283408    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283411    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283413    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283415    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283418    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283420    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283429    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283431    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283434    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283437    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283439    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283441    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283444    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283446    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283449    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283451    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283454    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 12:55:45.288676 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283457    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283459    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283462    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283464    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283467    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283469    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283472    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283475    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283477    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283480    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283483    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283485    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283487    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283490    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283493    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283495    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283498    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283500    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283503    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283506    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283508    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 12:55:45.289170 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283511    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 12:55:45.289691 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283519    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 12:55:45.289691 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283522    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 12:55:45.289691 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283527    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 12:55:45.289691 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283530    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 12:55:45.289691 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:45.283534    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 12:55:45.289691 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.283539    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 12:55:45.289691 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.283644    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 12:55:45.289691 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.285742    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 12:55:45.289691 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.286697    2582 server.go:1019] "Starting client certificate rotation"
Sep 11 12:55:45.289691 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.286797    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 12:55:45.289691 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.286876    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 12:55:45.291805 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.291773    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 12:55:45.293629 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.293612    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 12:55:45.302411 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.302396    2582 log.go:25] "Validated CRI v1 runtime API"
Sep 11 12:55:45.305298 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.305285    2582 log.go:25] "Validated CRI v1 image API"
Sep 11 12:55:45.308165 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.308149    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 12:55:45.311693 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.311674    2582 fs.go:135] Filesystem UUIDs: map[726e0166-be7f-4f06-98f8-2c4ba7a42ef5:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 98642185-4352-4bd7-9e93-79c147ea41d5:/dev/nvme0n1p3]
Sep 11 12:55:45.311757 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.311691    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 12:55:45.316480 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.316378    2582 manager.go:216] Machine: {Timestamp:2026-09-11 12:55:45.315107269 +0000 UTC m=+0.310419354 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099466 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2606d7cfec6370752c2351778875cf SystemUUID:ec2606d7-cfec-6370-752c-2351778875cf BootID:94a41e2c-735b-4939-80a2-4c194a731dba Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:01:42:c3:15:67 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:01:42:c3:15:67 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:f5:c4:35:2a:1d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 12:55:45.316480 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.316477    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 12:55:45.316579 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.316549    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 12:55:45.319135 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.319109    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 12:55:45.319269 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.319139    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-6.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 12:55:45.320102 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.320088    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 12:55:45.320211 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.320200    2582 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 12:55:45.320247 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.320216    2582 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 12:55:45.320247 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.320224    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 12:55:45.320874 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.320864    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 12:55:45.321910 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.321899    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 12:55:45.326891 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.326879    2582 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 12:55:45.327001 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.326992    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 12:55:45.327036 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.327019    2582 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 12:55:45.327036 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.327028    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 12:55:45.327086 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.327041    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 12:55:45.327086 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.327056    2582 kubelet.go:398] "Adding apiserver pod source"
Sep 11 12:55:45.327086 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.327068    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 12:55:45.327419 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.327407    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 12:55:45.327455 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.327427    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 12:55:45.329670 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.329653    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 12:55:45.332323 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.332307    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 12:55:45.332377 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.332342    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 12:55:45.333986 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.333974    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 12:55:45.334044 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.333990    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 12:55:45.334044 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.333997    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 12:55:45.334044 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.334002    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 12:55:45.334044 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.334014    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 12:55:45.334044 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.334023    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 12:55:45.334044 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.334045    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 12:55:45.334204 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.334051    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 12:55:45.334204 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.334058    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 12:55:45.334204 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.334063    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 12:55:45.334204 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.334072    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 12:55:45.334204 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.334077    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 12:55:45.334742 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.334732    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 12:55:45.334742 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.334741    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 12:55:45.337876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.337858    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-6.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 12:55:45.338754 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.338514    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-6.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 12:55:45.338837 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.338727    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 12:55:45.338899 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.338884    2582 server.go:1268] "Started kubelet"
Sep 11 12:55:45.339008 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.338960    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 12:55:45.339063 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.339033    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 12:55:45.339121 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.338936    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 12:55:45.339588 ip-10-0-143-6 systemd[1]: Started Kubernetes Kubelet.
Sep 11 12:55:45.340599 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.340572    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 12:55:45.340994 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.340961    2582 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 12:55:45.343659 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.343532    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 12:55:45.347490 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.346348    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-6.ec2.internal.18d4457ab7707715  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-6.ec2.internal,UID:ip-10-0-143-6.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-6.ec2.internal,},FirstTimestamp:2026-09-11 12:55:45.338857237 +0000 UTC m=+0.334169323,LastTimestamp:2026-09-11 12:55:45.338857237 +0000 UTC m=+0.334169323,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-6.ec2.internal,}"
Sep 11 12:55:45.348085 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.348063    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 12:55:45.348539 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.348518    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-68ws2"
Sep 11 12:55:45.348619 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.348585    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 12:55:45.348738 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.348723    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:45.349462 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.349318    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 12:55:45.349462 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.349320    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 12:55:45.349600 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.349467    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 12:55:45.349600 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.349558    2582 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 12:55:45.349600 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.349568    2582 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 12:55:45.350053 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.350030    2582 factory.go:55] Registering systemd factory
Sep 11 12:55:45.350053 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.350052    2582 factory.go:223] Registration of the systemd container factory successfully
Sep 11 12:55:45.350268 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.350238    2582 factory.go:153] Registering CRI-O factory
Sep 11 12:55:45.350329 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.350272    2582 factory.go:223] Registration of the crio container factory successfully
Sep 11 12:55:45.350329 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.350327    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 12:55:45.350399 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.350344    2582 factory.go:103] Registering Raw factory
Sep 11 12:55:45.350399 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.350381    2582 manager.go:1196] Started watching for new ooms in manager
Sep 11 12:55:45.350569 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.350522    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-6.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 12:55:45.350647 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.350632    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 12:55:45.351714 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.351697    2582 manager.go:349] Starting recovery of all containers
Sep 11 12:55:45.356073 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.356048    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-68ws2"
Sep 11 12:55:45.364152 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.364130    2582 manager.go:354] Recovery completed
Sep 11 12:55:45.367615 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.367591    2582 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 11 12:55:45.370752 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.370739    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 12:55:45.372413 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.372395    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 12:55:45.372467 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.372426    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 12:55:45.372467 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.372439    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID"
Sep 11 12:55:45.373117 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.373099    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 12:55:45.373117 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.373114    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 12:55:45.373216 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.373133    2582 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 12:55:45.373216 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.373172    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 12:55:45.374660 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.374645    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 12:55:45.374660 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.374668    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 12:55:45.374752 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.374678    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID"
Sep 11 12:55:45.375551 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.375539    2582 policy_none.go:49] "None policy: Start"
Sep 11 12:55:45.375592 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.375556    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 12:55:45.375592 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.375566    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 12:55:45.377420 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.377406    2582 policy_none.go:47] "Start"
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.412109    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.413257    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.413313    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.413340    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.413399    2582 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.415282    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.415558    2582 manager.go:337] "Starting Device Plugin manager"
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.415598    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.415610    2582 server.go:87] "Starting device plugin registration server"
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.415904    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.415916    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.416065    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.416129    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.416137    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.416720    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 12:55:45.425857 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.416795    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:45.514285 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.514253    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal"]
Sep 11 12:55:45.514367 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.514320    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 12:55:45.515269 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.515221    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 12:55:45.515269 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.515245    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 12:55:45.515269 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.515264    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID"
Sep 11 12:55:45.516313 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.516300    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 12:55:45.517012 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.516998    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 12:55:45.517071 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.517022    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 12:55:45.517071 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.517035    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID"
Sep 11 12:55:45.517071 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.517053    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.517360 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.517348    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 12:55:45.517477 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.517466    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.517523 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.517493    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 12:55:45.518052 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.518027    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 12:55:45.518131 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.518056    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 12:55:45.518131 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.518070    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID"
Sep 11 12:55:45.519462 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.519444    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 12:55:45.519556 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.519474    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 12:55:45.519556 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.519488    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID"
Sep 11 12:55:45.521482 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.521465    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.521572 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.521494    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 12:55:45.522200 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.522155    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 12:55:45.522200 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.522184    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 12:55:45.522200 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.522199    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID"
Sep 11 12:55:45.524417 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.524403    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.524465 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.524422    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-6.ec2.internal\": node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:45.539398 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.539381    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-6.ec2.internal\" not found" node="ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.540903 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.540885    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:45.544774 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.544759    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-6.ec2.internal\" not found" node="ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.550715 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.550698    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b4c3351bb8d511ff78a7dff4fde75874-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"b4c3351bb8d511ff78a7dff4fde75874\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.550820 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.550721    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4c3351bb8d511ff78a7dff4fde75874-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"b4c3351bb8d511ff78a7dff4fde75874\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.550820 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.550748    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf433a3f8ec5b3534116bb587dadd6f1-config\") pod \"kube-apiserver-proxy-ip-10-0-143-6.ec2.internal\" (UID: \"bf433a3f8ec5b3534116bb587dadd6f1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.641705 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.641686    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:45.651233 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.651211    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b4c3351bb8d511ff78a7dff4fde75874-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"b4c3351bb8d511ff78a7dff4fde75874\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.651233 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.651226    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b4c3351bb8d511ff78a7dff4fde75874-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"b4c3351bb8d511ff78a7dff4fde75874\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.651376 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.651260    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4c3351bb8d511ff78a7dff4fde75874-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"b4c3351bb8d511ff78a7dff4fde75874\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.651376 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.651286    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf433a3f8ec5b3534116bb587dadd6f1-config\") pod \"kube-apiserver-proxy-ip-10-0-143-6.ec2.internal\" (UID: \"bf433a3f8ec5b3534116bb587dadd6f1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.651376 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.651340    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bf433a3f8ec5b3534116bb587dadd6f1-config\") pod \"kube-apiserver-proxy-ip-10-0-143-6.ec2.internal\" (UID: \"bf433a3f8ec5b3534116bb587dadd6f1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.651376 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.651355    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4c3351bb8d511ff78a7dff4fde75874-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"b4c3351bb8d511ff78a7dff4fde75874\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.742426 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.742394    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:45.843191 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.843129    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:45.843191 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.843144    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.849474 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:45.849451    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal"
Sep 11 12:55:45.943672 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:45.943648    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:46.044151 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:46.044117    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:46.144714 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:46.144652    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:46.218837 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:46.218796    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 12:55:46.245089 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:46.245069    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:46.286598 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:46.286575    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 12:55:46.287050 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:46.286732    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 12:55:46.287050 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:46.286750    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 12:55:46.345524 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:46.345497    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:46.362713 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:46.362676    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 12:50:46 +0000 UTC" deadline="2028-06-03 16:12:18.483257226 +0000 UTC"
Sep 11 12:55:46.362713 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:46.362704    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15147h16m32.120557052s"
Sep 11 12:55:47.481145 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:47.481115    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4c3351bb8d511ff78a7dff4fde75874.slice/crio-30ff4184a977a16099adbba870a10efa81be1f5f6681915d78e640cdabc7f2b0 WatchSource:0}: Error finding container 30ff4184a977a16099adbba870a10efa81be1f5f6681915d78e640cdabc7f2b0: Status 404 returned error can't find the container with id 30ff4184a977a16099adbba870a10efa81be1f5f6681915d78e640cdabc7f2b0
Sep 11 12:55:47.481584 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:47.481554    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf433a3f8ec5b3534116bb587dadd6f1.slice/crio-b63742803dca4999590d84c30c3797bbfade088ec1fdfb30f01ef4e93dd824e8 WatchSource:0}: Error finding container b63742803dca4999590d84c30c3797bbfade088ec1fdfb30f01ef4e93dd824e8: Status 404 returned error can't find the container with id b63742803dca4999590d84c30c3797bbfade088ec1fdfb30f01ef4e93dd824e8
Sep 11 12:55:47.485994 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:47.485977    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:55:47.502164 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:47.502140    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:47.602677 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:47.602646    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:47.703214 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:47.703177    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:47.800750 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:47.800664    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 12:55:47.803514 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:47.803383    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:47.904195 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:47.904168    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found"
Sep 11 12:55:47.980865 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:47.980840    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 12:55:48.005163 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.004992    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal"
Sep 11 12:55:48.016578 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.016554    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 12:55:48.017665 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.017638    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal"
Sep 11 12:55:48.026338 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.026319    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 12:55:48.383584 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.383542    2582 apiserver.go:52] "Watching apiserver"
Sep 11 12:55:48.391992 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.391970    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 12:55:48.392376 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.392350    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal","openshift-cluster-node-tuning-operator/tuned-s49mh","openshift-image-registry/node-ca-sm29q","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal","openshift-multus/multus-additional-cni-plugins-d9z6v","openshift-multus/multus-tgmlr","openshift-network-diagnostics/network-check-target-6k8nb","kube-system/global-pull-secret-syncer-lbn4f","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8","openshift-dns/node-resolver-8lsb4","openshift-multus/network-metrics-daemon-g5nkq","openshift-network-operator/iptables-alerter-bpmlr","openshift-ovn-kubernetes/ovnkube-node-p9msj","kube-system/konnectivity-agent-zkjt7"]
Sep 11 12:55:48.394834 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.394815    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8lsb4"
Sep 11 12:55:48.397109 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.397088    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.397737 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.397713    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 12:55:48.397839 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.397716    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 12:55:48.397839 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.397790    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zg92r\""
Sep 11 12:55:48.399272 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.399243    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sm29q"
Sep 11 12:55:48.399798 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.399776    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 12:55:48.399877 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.399851    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 12:55:48.399965 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.399949    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 12:55:48.400161 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.400148    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 12:55:48.401244 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.401220    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 12:55:48.401330 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.401262    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gft5r\""
Sep 11 12:55:48.401330 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.401301    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 12:55:48.401657 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.401627    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zkjt7"
Sep 11 12:55:48.401765 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.401748    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 12:55:48.401829 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.401808    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 12:55:48.402158 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.402126    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 12:55:48.402249 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.402165    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vv7hk\""
Sep 11 12:55:48.403970 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.403950    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.404383 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.404147    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 12:55:48.404512 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.404494    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 12:55:48.404574 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.404550    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-l9r7w\""
Sep 11 12:55:48.404774 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.404758    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 12:55:48.406615 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.406596    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 12:55:48.406703 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.406642    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.406904 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.406866    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-twpsl\""
Sep 11 12:55:48.406984 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.406924    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 12:55:48.406984 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.406951    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 12:55:48.407082 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.407035    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 12:55:48.407289 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.407270    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 12:55:48.408954 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.408917    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-k7hcf\""
Sep 11 12:55:48.409040 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.408990    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.411604 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.411583    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 12:55:48.412581 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.411985    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:55:48.412581 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:48.412236    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:55:48.413251 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.413228    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-s76r9\""
Sep 11 12:55:48.413404 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.413389    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 12:55:48.413819 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.413801    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 12:55:48.413957 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.413939    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 12:55:48.415324 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415307    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.415438 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415307    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-kubelet\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.415495 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415411    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4af872d-253e-474d-ada0-07f8ac808485-host\") pod \"node-ca-sm29q\" (UID: \"b4af872d-253e-474d-ada0-07f8ac808485\") " pod="openshift-image-registry/node-ca-sm29q"
Sep 11 12:55:48.415495 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415460    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4af872d-253e-474d-ada0-07f8ac808485-serviceca\") pod \"node-ca-sm29q\" (UID: \"b4af872d-253e-474d-ada0-07f8ac808485\") " pod="openshift-image-registry/node-ca-sm29q"
Sep 11 12:55:48.415495 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415484    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-multus-cni-dir\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.415710 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415507    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-run-systemd\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.415710 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415532    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4j5d\" (UniqueName: \"kubernetes.io/projected/76e98787-2ece-4cfe-a0ef-57360c48e2a3-kube-api-access-l4j5d\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.415710 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415559    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/63cdffd2-b869-4eea-b71a-5c2d91da8429-cnibin\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.415710 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415584    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/63cdffd2-b869-4eea-b71a-5c2d91da8429-cni-binary-copy\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.415710 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415606    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/63cdffd2-b869-4eea-b71a-5c2d91da8429-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.415710 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415632    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-run-k8s-cni-cncf-io\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.415710 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415658    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-multus-conf-dir\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.415710 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415702    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbt4d\" (UniqueName: \"kubernetes.io/projected/8c08440e-84a0-40a3-b171-92220128ee4c-kube-api-access-fbt4d\") pod \"node-resolver-8lsb4\" (UID: \"8c08440e-84a0-40a3-b171-92220128ee4c\") " pod="openshift-dns/node-resolver-8lsb4"
Sep 11 12:55:48.416078 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415757    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-var-lib-openvswitch\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.416078 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415801    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-run-openvswitch\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.416078 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415827    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-run-ovn\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.416078 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415853    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-cni-netd\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.416078 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415875    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76e98787-2ece-4cfe-a0ef-57360c48e2a3-env-overrides\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.416078 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415900    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/63cdffd2-b869-4eea-b71a-5c2d91da8429-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.416078 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415924    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-run-netns\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.416078 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415954    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-etc-openvswitch\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.416078 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.415978    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-cni-bin\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.416078 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416008    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3d2a78d-a895-4b2f-b965-bd1cb39bdff2-agent-certs\") pod \"konnectivity-agent-zkjt7\" (UID: \"b3d2a78d-a895-4b2f-b965-bd1cb39bdff2\") " pod="kube-system/konnectivity-agent-zkjt7"
Sep 11 12:55:48.416078 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416030    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-cnibin\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.416078 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416065    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-var-lib-cni-bin\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416089    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-var-lib-kubelet\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416111    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88t4t\" (UniqueName: \"kubernetes.io/projected/01226bcb-c885-4780-bf4a-7dfd31f4e919-kube-api-access-88t4t\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416141    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c08440e-84a0-40a3-b171-92220128ee4c-tmp-dir\") pod \"node-resolver-8lsb4\" (UID: \"8c08440e-84a0-40a3-b171-92220128ee4c\") " pod="openshift-dns/node-resolver-8lsb4"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416163    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-node-log\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416188    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-log-socket\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416212    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416242    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3d2a78d-a895-4b2f-b965-bd1cb39bdff2-konnectivity-ca\") pod \"konnectivity-agent-zkjt7\" (UID: \"b3d2a78d-a895-4b2f-b965-bd1cb39bdff2\") " pod="kube-system/konnectivity-agent-zkjt7"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416267    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/63cdffd2-b869-4eea-b71a-5c2d91da8429-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416304    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01226bcb-c885-4780-bf4a-7dfd31f4e919-cni-binary-copy\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416335    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-multus-socket-dir-parent\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416372    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8c08440e-84a0-40a3-b171-92220128ee4c-hosts-file\") pod \"node-resolver-8lsb4\" (UID: \"8c08440e-84a0-40a3-b171-92220128ee4c\") " pod="openshift-dns/node-resolver-8lsb4"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416399    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416490    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-system-cni-dir\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416517    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-run-netns\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416543    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-etc-kubernetes\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.416638 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416566    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-slash\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.417406 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416586    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76e98787-2ece-4cfe-a0ef-57360c48e2a3-ovn-node-metrics-cert\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.417406 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416620    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8wmw\" (UniqueName: \"kubernetes.io/projected/b4af872d-253e-474d-ada0-07f8ac808485-kube-api-access-l8wmw\") pod \"node-ca-sm29q\" (UID: \"b4af872d-253e-474d-ada0-07f8ac808485\") " pod="openshift-image-registry/node-ca-sm29q"
Sep 11 12:55:48.417406 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416642    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63cdffd2-b869-4eea-b71a-5c2d91da8429-system-cni-dir\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.417406 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416664    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/01226bcb-c885-4780-bf4a-7dfd31f4e919-multus-daemon-config\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.417406 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416701    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-run-multus-certs\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.417406 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416733    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-systemd-units\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.417406 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416763    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76e98787-2ece-4cfe-a0ef-57360c48e2a3-ovnkube-config\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.417406 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416777    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76e98787-2ece-4cfe-a0ef-57360c48e2a3-ovnkube-script-lib\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.417406 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416791    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdqhj\" (UniqueName: \"kubernetes.io/projected/63cdffd2-b869-4eea-b71a-5c2d91da8429-kube-api-access-jdqhj\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.417406 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416804    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-var-lib-cni-multus\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.417406 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.416858    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-hostroot\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.417932 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.417743    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 12:55:48.417932 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.417794    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6zb8g\""
Sep 11 12:55:48.417932 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.417806    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 12:55:48.417932 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.417796    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:48.418112 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:48.417994    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:55:48.418305 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.418286    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 12:55:48.420319 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.420301    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:55:48.420411 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:48.420367    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:55:48.422562 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.422545    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bpmlr"
Sep 11 12:55:48.424888 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.424867    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 12:55:48.425037 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.425021    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-h88d9\""
Sep 11 12:55:48.425118 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.425039    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 12:55:48.425118 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.425028    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 12:55:48.442159 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.442143    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-47ngr"
Sep 11 12:55:48.450561 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.450546    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-47ngr"
Sep 11 12:55:48.473916 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.473881    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" event={"ID":"b4c3351bb8d511ff78a7dff4fde75874","Type":"ContainerStarted","Data":"30ff4184a977a16099adbba870a10efa81be1f5f6681915d78e640cdabc7f2b0"}
Sep 11 12:55:48.475110 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.475078    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" event={"ID":"bf433a3f8ec5b3534116bb587dadd6f1","Type":"ContainerStarted","Data":"b63742803dca4999590d84c30c3797bbfade088ec1fdfb30f01ef4e93dd824e8"}
Sep 11 12:55:48.506624 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.506603    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 12:55:48.517114 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517097    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-tmp\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.517205 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517122    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eee29df6-f28a-485c-9e0b-e53a3b6d039e-host-slash\") pod \"iptables-alerter-bpmlr\" (UID: \"eee29df6-f28a-485c-9e0b-e53a3b6d039e\") " pod="openshift-network-operator/iptables-alerter-bpmlr"
Sep 11 12:55:48.517205 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517148    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8wmw\" (UniqueName: \"kubernetes.io/projected/b4af872d-253e-474d-ada0-07f8ac808485-kube-api-access-l8wmw\") pod \"node-ca-sm29q\" (UID: \"b4af872d-253e-474d-ada0-07f8ac808485\") " pod="openshift-image-registry/node-ca-sm29q"
Sep 11 12:55:48.517205 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517179    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63cdffd2-b869-4eea-b71a-5c2d91da8429-system-cni-dir\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.517412 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517299    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63cdffd2-b869-4eea-b71a-5c2d91da8429-system-cni-dir\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.517412 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517341    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-run-multus-certs\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.517412 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517374    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-modprobe-d\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.517412 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517375    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-run-multus-certs\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.517412 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517402    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-systemd\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.517664 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517461    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-dbus\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:48.517664 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517500    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcngr\" (UniqueName: \"kubernetes.io/projected/eee29df6-f28a-485c-9e0b-e53a3b6d039e-kube-api-access-kcngr\") pod \"iptables-alerter-bpmlr\" (UID: \"eee29df6-f28a-485c-9e0b-e53a3b6d039e\") " pod="openshift-network-operator/iptables-alerter-bpmlr"
Sep 11 12:55:48.517664 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517525    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-var-lib-cni-multus\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.517664 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517549    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp9zg\" (UniqueName: \"kubernetes.io/projected/4c721b01-f754-4b83-87fe-94689fb629c9-kube-api-access-zp9zg\") pod \"network-metrics-daemon-g5nkq\" (UID: \"4c721b01-f754-4b83-87fe-94689fb629c9\") " pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:55:48.517664 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517585    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-var-lib-cni-multus\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.517664 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517581    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-kubelet\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.517664 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517633    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-kubelet\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.517954 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517691    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4af872d-253e-474d-ada0-07f8ac808485-host\") pod \"node-ca-sm29q\" (UID: \"b4af872d-253e-474d-ada0-07f8ac808485\") " pod="openshift-image-registry/node-ca-sm29q"
Sep 11 12:55:48.517954 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517722    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4af872d-253e-474d-ada0-07f8ac808485-host\") pod \"node-ca-sm29q\" (UID: \"b4af872d-253e-474d-ada0-07f8ac808485\") " pod="openshift-image-registry/node-ca-sm29q"
Sep 11 12:55:48.517954 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517745    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4af872d-253e-474d-ada0-07f8ac808485-serviceca\") pod \"node-ca-sm29q\" (UID: \"b4af872d-253e-474d-ada0-07f8ac808485\") " pod="openshift-image-registry/node-ca-sm29q"
Sep 11 12:55:48.517954 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517813    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-multus-cni-dir\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.517954 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517890    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-multus-cni-dir\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.517954 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517891    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-sysctl-d\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.517954 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517931    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9r54\" (UniqueName: \"kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54\") pod \"network-check-target-6k8nb\" (UID: \"0a1899d1-b90c-4b2f-b49c-06c3a2e3533d\") " pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:55:48.518273 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.517963    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-run-systemd\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.518273 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518000    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/63cdffd2-b869-4eea-b71a-5c2d91da8429-cnibin\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.518273 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518022    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/63cdffd2-b869-4eea-b71a-5c2d91da8429-cni-binary-copy\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.518273 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518036    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-run-systemd\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.518273 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/63cdffd2-b869-4eea-b71a-5c2d91da8429-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.518273 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518095    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/63cdffd2-b869-4eea-b71a-5c2d91da8429-cnibin\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.518273 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518111    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-socket-dir\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.518273 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518159    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-var-lib-openvswitch\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.518273 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518192    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4af872d-253e-474d-ada0-07f8ac808485-serviceca\") pod \"node-ca-sm29q\" (UID: \"b4af872d-253e-474d-ada0-07f8ac808485\") " pod="openshift-image-registry/node-ca-sm29q"
Sep 11 12:55:48.518273 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518210    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-run-ovn\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.518273 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518247    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-cni-netd\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.518273 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518261    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-var-lib-openvswitch\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.518273 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518265    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/63cdffd2-b869-4eea-b71a-5c2d91da8429-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.518273 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518277    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-run-ovn\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518318    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-cni-netd\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518348    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-cni-bin\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518374    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-cnibin\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518396    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-var-lib-cni-bin\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518436    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-node-log\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518459    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-log-socket\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518481    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3d2a78d-a895-4b2f-b965-bd1cb39bdff2-konnectivity-ca\") pod \"konnectivity-agent-zkjt7\" (UID: \"b3d2a78d-a895-4b2f-b965-bd1cb39bdff2\") " pod="kube-system/konnectivity-agent-zkjt7"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518526    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01226bcb-c885-4780-bf4a-7dfd31f4e919-cni-binary-copy\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518554    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-multus-socket-dir-parent\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518579    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-sys\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518602    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n776z\" (UniqueName: \"kubernetes.io/projected/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-kube-api-access-n776z\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518631    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8c08440e-84a0-40a3-b171-92220128ee4c-hosts-file\") pod \"node-resolver-8lsb4\" (UID: \"8c08440e-84a0-40a3-b171-92220128ee4c\") " pod="openshift-dns/node-resolver-8lsb4"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518655    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-system-cni-dir\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518679    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-run-netns\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518683    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/63cdffd2-b869-4eea-b71a-5c2d91da8429-cni-binary-copy\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518702    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-etc-selinux\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.518876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518730    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-lib-modules\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518758    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-slash\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518790    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/63cdffd2-b869-4eea-b71a-5c2d91da8429-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518817    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76e98787-2ece-4cfe-a0ef-57360c48e2a3-ovn-node-metrics-cert\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518841    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/01226bcb-c885-4780-bf4a-7dfd31f4e919-multus-daemon-config\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518857    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-cnibin\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518885    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.518910    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-sys-fs\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519358    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-system-cni-dir\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519368    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-node-log\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519368    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-log-socket\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519462    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-var-lib-cni-bin\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519522    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/63cdffd2-b869-4eea-b71a-5c2d91da8429-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519550    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-sysctl-conf\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519555    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-multus-socket-dir-parent\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519602    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-run\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.519625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519614    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-run-netns\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.520367 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519626    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3d2a78d-a895-4b2f-b965-bd1cb39bdff2-konnectivity-ca\") pod \"konnectivity-agent-zkjt7\" (UID: \"b3d2a78d-a895-4b2f-b965-bd1cb39bdff2\") " pod="kube-system/konnectivity-agent-zkjt7"
Sep 11 12:55:48.520367 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519707    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-systemd-units\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.520367 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519785    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-slash\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.520367 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519802    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-systemd-units\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.520367 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519882    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76e98787-2ece-4cfe-a0ef-57360c48e2a3-ovnkube-config\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.520367 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519909    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01226bcb-c885-4780-bf4a-7dfd31f4e919-cni-binary-copy\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.520367 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.519958    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8c08440e-84a0-40a3-b171-92220128ee4c-hosts-file\") pod \"node-resolver-8lsb4\" (UID: \"8c08440e-84a0-40a3-b171-92220128ee4c\") " pod="openshift-dns/node-resolver-8lsb4"
Sep 11 12:55:48.520367 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.520003    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76e98787-2ece-4cfe-a0ef-57360c48e2a3-ovnkube-script-lib\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.520367 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.520089    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-cni-bin\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.520367 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.520141    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdqhj\" (UniqueName: \"kubernetes.io/projected/63cdffd2-b869-4eea-b71a-5c2d91da8429-kube-api-access-jdqhj\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.520367 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.520235    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/01226bcb-c885-4780-bf4a-7dfd31f4e919-multus-daemon-config\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.520886 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.520547    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76e98787-2ece-4cfe-a0ef-57360c48e2a3-ovnkube-config\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.520960    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.521015    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-hostroot\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.521067    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5q56\" (UniqueName: \"kubernetes.io/projected/3fee639e-a752-4a3c-ba28-503b0dd550af-kube-api-access-c5q56\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.521166    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-sysconfig\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.521207    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-host\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.521243    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eee29df6-f28a-485c-9e0b-e53a3b6d039e-iptables-alerter-script\") pod \"iptables-alerter-bpmlr\" (UID: \"eee29df6-f28a-485c-9e0b-e53a3b6d039e\") " pod="openshift-network-operator/iptables-alerter-bpmlr"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.521276    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4j5d\" (UniqueName: \"kubernetes.io/projected/76e98787-2ece-4cfe-a0ef-57360c48e2a3-kube-api-access-l4j5d\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.521444    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-run-k8s-cni-cncf-io\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.521468    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76e98787-2ece-4cfe-a0ef-57360c48e2a3-ovnkube-script-lib\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.521486    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-multus-conf-dir\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.521522    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-device-dir\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.521569    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-hostroot\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.521562    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-kubernetes\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.521847    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-run-k8s-cni-cncf-io\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.521875    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-var-lib-kubelet\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.522049    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:48.522150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.522059    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-multus-conf-dir\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.522993 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.522109    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs\") pod \"network-metrics-daemon-g5nkq\" (UID: \"4c721b01-f754-4b83-87fe-94689fb629c9\") " pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:55:48.523358 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523085    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbt4d\" (UniqueName: \"kubernetes.io/projected/8c08440e-84a0-40a3-b171-92220128ee4c-kube-api-access-fbt4d\") pod \"node-resolver-8lsb4\" (UID: \"8c08440e-84a0-40a3-b171-92220128ee4c\") " pod="openshift-dns/node-resolver-8lsb4"
Sep 11 12:55:48.523358 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523146    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-run-openvswitch\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.523358 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523175    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76e98787-2ece-4cfe-a0ef-57360c48e2a3-env-overrides\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.523358 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523246    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-var-lib-kubelet\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.523358 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523275    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88t4t\" (UniqueName: \"kubernetes.io/projected/01226bcb-c885-4780-bf4a-7dfd31f4e919-kube-api-access-88t4t\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.523358 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523303    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-registration-dir\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.523358 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523327    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-run-openvswitch\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.523728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523387    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-tuned\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.523728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523444    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-run-netns\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.523728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523470    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-etc-openvswitch\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.523728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523499    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3d2a78d-a895-4b2f-b965-bd1cb39bdff2-agent-certs\") pod \"konnectivity-agent-zkjt7\" (UID: \"b3d2a78d-a895-4b2f-b965-bd1cb39bdff2\") " pod="kube-system/konnectivity-agent-zkjt7"
Sep 11 12:55:48.523728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523548    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-kubelet-config\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:48.523728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523578    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c08440e-84a0-40a3-b171-92220128ee4c-tmp-dir\") pod \"node-resolver-8lsb4\" (UID: \"8c08440e-84a0-40a3-b171-92220128ee4c\") " pod="openshift-dns/node-resolver-8lsb4"
Sep 11 12:55:48.523728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523673    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.523728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523704    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/63cdffd2-b869-4eea-b71a-5c2d91da8429-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.523728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523730    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-etc-kubernetes\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.524107 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523759    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.524107 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.523857    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.524107 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.524095    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c08440e-84a0-40a3-b171-92220128ee4c-tmp-dir\") pod \"node-resolver-8lsb4\" (UID: \"8c08440e-84a0-40a3-b171-92220128ee4c\") " pod="openshift-dns/node-resolver-8lsb4"
Sep 11 12:55:48.524244 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.524162    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-run-netns\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.524297 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.524261    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-etc-openvswitch\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.524710 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.524453    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76e98787-2ece-4cfe-a0ef-57360c48e2a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.524710 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.524507    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-host-var-lib-kubelet\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.524710 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.524570    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01226bcb-c885-4780-bf4a-7dfd31f4e919-etc-kubernetes\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.524710 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.524655    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76e98787-2ece-4cfe-a0ef-57360c48e2a3-env-overrides\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.525846 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.525812    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/63cdffd2-b869-4eea-b71a-5c2d91da8429-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.526185 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.526163    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76e98787-2ece-4cfe-a0ef-57360c48e2a3-ovn-node-metrics-cert\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.526984 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.526963    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3d2a78d-a895-4b2f-b965-bd1cb39bdff2-agent-certs\") pod \"konnectivity-agent-zkjt7\" (UID: \"b3d2a78d-a895-4b2f-b965-bd1cb39bdff2\") " pod="kube-system/konnectivity-agent-zkjt7"
Sep 11 12:55:48.527174 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.527159    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8wmw\" (UniqueName: \"kubernetes.io/projected/b4af872d-253e-474d-ada0-07f8ac808485-kube-api-access-l8wmw\") pod \"node-ca-sm29q\" (UID: \"b4af872d-253e-474d-ada0-07f8ac808485\") " pod="openshift-image-registry/node-ca-sm29q"
Sep 11 12:55:48.530344 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.530302    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdqhj\" (UniqueName: \"kubernetes.io/projected/63cdffd2-b869-4eea-b71a-5c2d91da8429-kube-api-access-jdqhj\") pod \"multus-additional-cni-plugins-d9z6v\" (UID: \"63cdffd2-b869-4eea-b71a-5c2d91da8429\") " pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.531036 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.531010    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4j5d\" (UniqueName: \"kubernetes.io/projected/76e98787-2ece-4cfe-a0ef-57360c48e2a3-kube-api-access-l4j5d\") pod \"ovnkube-node-p9msj\" (UID: \"76e98787-2ece-4cfe-a0ef-57360c48e2a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.531716 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.531688    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88t4t\" (UniqueName: \"kubernetes.io/projected/01226bcb-c885-4780-bf4a-7dfd31f4e919-kube-api-access-88t4t\") pod \"multus-tgmlr\" (UID: \"01226bcb-c885-4780-bf4a-7dfd31f4e919\") " pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.532040 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.532015    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbt4d\" (UniqueName: \"kubernetes.io/projected/8c08440e-84a0-40a3-b171-92220128ee4c-kube-api-access-fbt4d\") pod \"node-resolver-8lsb4\" (UID: \"8c08440e-84a0-40a3-b171-92220128ee4c\") " pod="openshift-dns/node-resolver-8lsb4"
Sep 11 12:55:48.624204 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624181    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-sys\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.624302 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624207    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n776z\" (UniqueName: \"kubernetes.io/projected/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-kube-api-access-n776z\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.624302 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624224    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-etc-selinux\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.624302 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-lib-modules\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.624481 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624305    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-sys\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.624481 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624402    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-etc-selinux\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.624481 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624433    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.624481 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624456    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-lib-modules\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.624481 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624469    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-sys-fs\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624490    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-sysctl-conf\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624502    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-sys-fs\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624504    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-run\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624499    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624538    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5q56\" (UniqueName: \"kubernetes.io/projected/3fee639e-a752-4a3c-ba28-503b0dd550af-kube-api-access-c5q56\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624561    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-sysconfig\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624581    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-host\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624603    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-run\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624603    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eee29df6-f28a-485c-9e0b-e53a3b6d039e-iptables-alerter-script\") pod \"iptables-alerter-bpmlr\" (UID: \"eee29df6-f28a-485c-9e0b-e53a3b6d039e\") " pod="openshift-network-operator/iptables-alerter-bpmlr"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624637    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-sysctl-conf\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624642    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-sysconfig\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624656    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-host\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624647    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-device-dir\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624682    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-device-dir\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624707    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-kubernetes\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.624728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624732    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-var-lib-kubelet\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624782    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-var-lib-kubelet\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624786    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-kubernetes\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624779    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624841    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs\") pod \"network-metrics-daemon-g5nkq\" (UID: \"4c721b01-f754-4b83-87fe-94689fb629c9\") " pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:48.624868    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624872    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-registration-dir\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624895    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-tuned\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:48.624937    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret podName:da6d1a19-06a1-4a7a-bbc2-593b438eeb41 nodeName:}" failed. No retries permitted until 2026-09-11 12:55:49.12491763 +0000 UTC m=+3.064618961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret") pod "global-pull-secret-syncer-lbn4f" (UID: "da6d1a19-06a1-4a7a-bbc2-593b438eeb41") : object "kube-system"/"original-pull-secret" not registered
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624954    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-kubelet-config\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624956    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-registration-dir\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:48.624936    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.624985    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-tmp\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625006    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eee29df6-f28a-485c-9e0b-e53a3b6d039e-host-slash\") pod \"iptables-alerter-bpmlr\" (UID: \"eee29df6-f28a-485c-9e0b-e53a3b6d039e\") " pod="openshift-network-operator/iptables-alerter-bpmlr"
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:48.625037    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs podName:4c721b01-f754-4b83-87fe-94689fb629c9 nodeName:}" failed. No retries permitted until 2026-09-11 12:55:49.125021364 +0000 UTC m=+3.064722684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs") pod "network-metrics-daemon-g5nkq" (UID: "4c721b01-f754-4b83-87fe-94689fb629c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625048    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eee29df6-f28a-485c-9e0b-e53a3b6d039e-host-slash\") pod \"iptables-alerter-bpmlr\" (UID: \"eee29df6-f28a-485c-9e0b-e53a3b6d039e\") " pod="openshift-network-operator/iptables-alerter-bpmlr"
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-modprobe-d\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.625466 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625086    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-systemd\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.626156 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625109    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-dbus\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:48.626156 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625114    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eee29df6-f28a-485c-9e0b-e53a3b6d039e-iptables-alerter-script\") pod \"iptables-alerter-bpmlr\" (UID: \"eee29df6-f28a-485c-9e0b-e53a3b6d039e\") " pod="openshift-network-operator/iptables-alerter-bpmlr"
Sep 11 12:55:48.626156 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625057    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-kubelet-config\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:48.626156 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625133    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcngr\" (UniqueName: \"kubernetes.io/projected/eee29df6-f28a-485c-9e0b-e53a3b6d039e-kube-api-access-kcngr\") pod \"iptables-alerter-bpmlr\" (UID: \"eee29df6-f28a-485c-9e0b-e53a3b6d039e\") " pod="openshift-network-operator/iptables-alerter-bpmlr"
Sep 11 12:55:48.626156 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625164    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp9zg\" (UniqueName: \"kubernetes.io/projected/4c721b01-f754-4b83-87fe-94689fb629c9-kube-api-access-zp9zg\") pod \"network-metrics-daemon-g5nkq\" (UID: \"4c721b01-f754-4b83-87fe-94689fb629c9\") " pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:55:48.626156 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625196    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-sysctl-d\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.626156 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625246    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9r54\" (UniqueName: \"kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54\") pod \"network-check-target-6k8nb\" (UID: \"0a1899d1-b90c-4b2f-b49c-06c3a2e3533d\") " pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:55:48.626156 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625273    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-socket-dir\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.626156 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625289    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-systemd\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.626156 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625288    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-modprobe-d\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.626156 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625316    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-dbus\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:48.626156 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625413    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3fee639e-a752-4a3c-ba28-503b0dd550af-socket-dir\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.626156 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.625438    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-sysctl-d\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.627207 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.627185    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-tmp\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.627446 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.627409    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-etc-tuned\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.633734 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:48.633683    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 12:55:48.633734 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:48.633705    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 12:55:48.633734 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:48.633717    2582 projected.go:196] Error preparing data for projected volume kube-api-access-t9r54 for pod openshift-network-diagnostics/network-check-target-6k8nb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:55:48.633927 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:48.633771    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54 podName:0a1899d1-b90c-4b2f-b49c-06c3a2e3533d nodeName:}" failed. No retries permitted until 2026-09-11 12:55:49.133756245 +0000 UTC m=+3.073457576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-t9r54" (UniqueName: "kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54") pod "network-check-target-6k8nb" (UID: "0a1899d1-b90c-4b2f-b49c-06c3a2e3533d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:55:48.635819 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.635797    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n776z\" (UniqueName: \"kubernetes.io/projected/8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27-kube-api-access-n776z\") pod \"tuned-s49mh\" (UID: \"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27\") " pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.636609 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.636591    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5q56\" (UniqueName: \"kubernetes.io/projected/3fee639e-a752-4a3c-ba28-503b0dd550af-kube-api-access-c5q56\") pod \"aws-ebs-csi-driver-node-qbvd8\" (UID: \"3fee639e-a752-4a3c-ba28-503b0dd550af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.636693 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.636624    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp9zg\" (UniqueName: \"kubernetes.io/projected/4c721b01-f754-4b83-87fe-94689fb629c9-kube-api-access-zp9zg\") pod \"network-metrics-daemon-g5nkq\" (UID: \"4c721b01-f754-4b83-87fe-94689fb629c9\") " pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:55:48.637062 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.637042    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcngr\" (UniqueName: \"kubernetes.io/projected/eee29df6-f28a-485c-9e0b-e53a3b6d039e-kube-api-access-kcngr\") pod \"iptables-alerter-bpmlr\" (UID: \"eee29df6-f28a-485c-9e0b-e53a3b6d039e\") " pod="openshift-network-operator/iptables-alerter-bpmlr"
Sep 11 12:55:48.680017 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.679998    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 12:55:48.706099 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.706079    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8lsb4"
Sep 11 12:55:48.716054 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.716033    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:55:48.726066 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.726044    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sm29q"
Sep 11 12:55:48.730912 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.730895    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zkjt7"
Sep 11 12:55:48.736856 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.736838    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d9z6v"
Sep 11 12:55:48.743449 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.743432    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tgmlr"
Sep 11 12:55:48.750272 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.750256    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8"
Sep 11 12:55:48.756034 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.756019    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s49mh"
Sep 11 12:55:48.763050 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:48.763029    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bpmlr"
Sep 11 12:55:49.128159 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.128123    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:49.128159 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.128160    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs\") pod \"network-metrics-daemon-g5nkq\" (UID: \"4c721b01-f754-4b83-87fe-94689fb629c9\") " pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:55:49.128378 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:49.128268    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:55:49.128378 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:49.128291    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 12:55:49.128378 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:49.128329    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs podName:4c721b01-f754-4b83-87fe-94689fb629c9 nodeName:}" failed. No retries permitted until 2026-09-11 12:55:50.128313792 +0000 UTC m=+4.068015123 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs") pod "network-metrics-daemon-g5nkq" (UID: "4c721b01-f754-4b83-87fe-94689fb629c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:55:49.128378 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:49.128355    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret podName:da6d1a19-06a1-4a7a-bbc2-593b438eeb41 nodeName:}" failed. No retries permitted until 2026-09-11 12:55:50.128336762 +0000 UTC m=+4.068038077 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret") pod "global-pull-secret-syncer-lbn4f" (UID: "da6d1a19-06a1-4a7a-bbc2-593b438eeb41") : object "kube-system"/"original-pull-secret" not registered
Sep 11 12:55:49.229330 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.229295    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9r54\" (UniqueName: \"kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54\") pod \"network-check-target-6k8nb\" (UID: \"0a1899d1-b90c-4b2f-b49c-06c3a2e3533d\") " pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:55:49.229488 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:49.229468    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 12:55:49.229488 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:49.229486    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 12:55:49.229613 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:49.229498    2582 projected.go:196] Error preparing data for projected volume kube-api-access-t9r54 for pod openshift-network-diagnostics/network-check-target-6k8nb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:55:49.229613 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:49.229553    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54 podName:0a1899d1-b90c-4b2f-b49c-06c3a2e3533d nodeName:}" failed. No retries permitted until 2026-09-11 12:55:50.229539306 +0000 UTC m=+4.169240617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9r54" (UniqueName: "kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54") pod "network-check-target-6k8nb" (UID: "0a1899d1-b90c-4b2f-b49c-06c3a2e3533d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:55:49.362868 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:49.362835    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3d2a78d_a895_4b2f_b965_bd1cb39bdff2.slice/crio-83a6a27c92dfa2a4fdf3f6a857339647506b6649438a18bce14022efc94a45ad WatchSource:0}: Error finding container 83a6a27c92dfa2a4fdf3f6a857339647506b6649438a18bce14022efc94a45ad: Status 404 returned error can't find the container with id 83a6a27c92dfa2a4fdf3f6a857339647506b6649438a18bce14022efc94a45ad
Sep 11 12:55:49.364616 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:49.364592    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4af872d_253e_474d_ada0_07f8ac808485.slice/crio-1cb9be61930d3d561a6dd9ba57e82f9f14f8533b49471922753e04d3c50d1e8c WatchSource:0}: Error finding container 1cb9be61930d3d561a6dd9ba57e82f9f14f8533b49471922753e04d3c50d1e8c: Status 404 returned error can't find the container with id 1cb9be61930d3d561a6dd9ba57e82f9f14f8533b49471922753e04d3c50d1e8c
Sep 11 12:55:49.366251 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:49.366127    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f2bd220_7fb1_4c41_bbc4_28f57fdf9a27.slice/crio-6f8dc41d9cd47ba2456e08fefe286071c015c1b9ce7b1ad8215c3d68a27b43d5 WatchSource:0}: Error finding container 6f8dc41d9cd47ba2456e08fefe286071c015c1b9ce7b1ad8215c3d68a27b43d5: Status 404 returned error can't find the container with id 6f8dc41d9cd47ba2456e08fefe286071c015c1b9ce7b1ad8215c3d68a27b43d5
Sep 11 12:55:49.366936 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:49.366908    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee29df6_f28a_485c_9e0b_e53a3b6d039e.slice/crio-a058afa7836258b5cba83d0ee95bcf650fc6dfe2af35063fbce49acffc3f68cb WatchSource:0}: Error finding container a058afa7836258b5cba83d0ee95bcf650fc6dfe2af35063fbce49acffc3f68cb: Status 404 returned error can't find the container with id a058afa7836258b5cba83d0ee95bcf650fc6dfe2af35063fbce49acffc3f68cb
Sep 11 12:55:49.368104 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:49.368057    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63cdffd2_b869_4eea_b71a_5c2d91da8429.slice/crio-df25908ac7258e43da427947d374f55066d86b65f6cc80e30d773a10d1ea5188 WatchSource:0}: Error finding container df25908ac7258e43da427947d374f55066d86b65f6cc80e30d773a10d1ea5188: Status 404 returned error can't find the container with id df25908ac7258e43da427947d374f55066d86b65f6cc80e30d773a10d1ea5188
Sep 11 12:55:49.370149 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:49.370130    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76e98787_2ece_4cfe_a0ef_57360c48e2a3.slice/crio-29d7a03dd1aaba67d8f630097bb4b6d6382969453f3da242b40e3436e120f4c0 WatchSource:0}: Error finding container 29d7a03dd1aaba67d8f630097bb4b6d6382969453f3da242b40e3436e120f4c0: Status 404 returned error can't find the container with id 29d7a03dd1aaba67d8f630097bb4b6d6382969453f3da242b40e3436e120f4c0
Sep 11 12:55:49.371528 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:49.371488    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fee639e_a752_4a3c_ba28_503b0dd550af.slice/crio-3a0663f73ba2f4e3f03cd77b9bbb3ef9fff7caa7cf3dbc18a5f8df1b0ac40de4 WatchSource:0}: Error finding container 3a0663f73ba2f4e3f03cd77b9bbb3ef9fff7caa7cf3dbc18a5f8df1b0ac40de4: Status 404 returned error can't find the container with id 3a0663f73ba2f4e3f03cd77b9bbb3ef9fff7caa7cf3dbc18a5f8df1b0ac40de4
Sep 11 12:55:49.372443 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:49.372394    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c08440e_84a0_40a3_b171_92220128ee4c.slice/crio-74bae29e60e6e4d576740761340cc60314cba9da6e113d02a18c4d4ae8f40faa WatchSource:0}: Error finding container 74bae29e60e6e4d576740761340cc60314cba9da6e113d02a18c4d4ae8f40faa: Status 404 returned error can't find the container with id 74bae29e60e6e4d576740761340cc60314cba9da6e113d02a18c4d4ae8f40faa
Sep 11 12:55:49.374063 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:55:49.373867    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01226bcb_c885_4780_bf4a_7dfd31f4e919.slice/crio-afd1c046a1dff037692926a4d51cd7068237a42dc7c7dd68c3c0abce3f8d1b31 WatchSource:0}: Error finding container afd1c046a1dff037692926a4d51cd7068237a42dc7c7dd68c3c0abce3f8d1b31: Status 404 returned error can't find the container with id afd1c046a1dff037692926a4d51cd7068237a42dc7c7dd68c3c0abce3f8d1b31
Sep 11 12:55:49.453872 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.453846    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 12:50:48 +0000 UTC" deadline="2028-03-14 06:47:33.880085622 +0000 UTC"
Sep 11 12:55:49.453872 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.453871    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13193h51m44.426218182s"
Sep 11 12:55:49.477993 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.477965    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8lsb4" event={"ID":"8c08440e-84a0-40a3-b171-92220128ee4c","Type":"ContainerStarted","Data":"74bae29e60e6e4d576740761340cc60314cba9da6e113d02a18c4d4ae8f40faa"}
Sep 11 12:55:49.481251 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.481230    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tgmlr" event={"ID":"01226bcb-c885-4780-bf4a-7dfd31f4e919","Type":"ContainerStarted","Data":"afd1c046a1dff037692926a4d51cd7068237a42dc7c7dd68c3c0abce3f8d1b31"}
Sep 11 12:55:49.483857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.483838    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj" event={"ID":"76e98787-2ece-4cfe-a0ef-57360c48e2a3","Type":"ContainerStarted","Data":"29d7a03dd1aaba67d8f630097bb4b6d6382969453f3da242b40e3436e120f4c0"}
Sep 11 12:55:49.486314 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.486294    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bpmlr" event={"ID":"eee29df6-f28a-485c-9e0b-e53a3b6d039e","Type":"ContainerStarted","Data":"a058afa7836258b5cba83d0ee95bcf650fc6dfe2af35063fbce49acffc3f68cb"}
Sep 11 12:55:49.487866 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.487845    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s49mh" event={"ID":"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27","Type":"ContainerStarted","Data":"6f8dc41d9cd47ba2456e08fefe286071c015c1b9ce7b1ad8215c3d68a27b43d5"}
Sep 11 12:55:49.489925 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.489900    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zkjt7" event={"ID":"b3d2a78d-a895-4b2f-b965-bd1cb39bdff2","Type":"ContainerStarted","Data":"83a6a27c92dfa2a4fdf3f6a857339647506b6649438a18bce14022efc94a45ad"}
Sep 11 12:55:49.494465 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.494446    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8" event={"ID":"3fee639e-a752-4a3c-ba28-503b0dd550af","Type":"ContainerStarted","Data":"3a0663f73ba2f4e3f03cd77b9bbb3ef9fff7caa7cf3dbc18a5f8df1b0ac40de4"}
Sep 11 12:55:49.496007 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.495978    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9z6v" event={"ID":"63cdffd2-b869-4eea-b71a-5c2d91da8429","Type":"ContainerStarted","Data":"df25908ac7258e43da427947d374f55066d86b65f6cc80e30d773a10d1ea5188"}
Sep 11 12:55:49.500348 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.500324    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sm29q" event={"ID":"b4af872d-253e-474d-ada0-07f8ac808485","Type":"ContainerStarted","Data":"1cb9be61930d3d561a6dd9ba57e82f9f14f8533b49471922753e04d3c50d1e8c"}
Sep 11 12:55:49.502651 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.502633    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" event={"ID":"bf433a3f8ec5b3534116bb587dadd6f1","Type":"ContainerStarted","Data":"319d85d11a2552b436c4669d50e652016905ec108e15c88083995272fb6fc9d3"}
Sep 11 12:55:49.517944 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:49.517907    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" podStartSLOduration=1.5178955429999998 podStartE2EDuration="1.517895543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:55:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:55:49.517757394 +0000 UTC m=+3.457458723" watchObservedRunningTime="2026-09-11 12:55:49.517895543 +0000 UTC m=+3.457596876"
Sep 11 12:55:50.136445 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:50.136394    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:50.136631 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:50.136458    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs\") pod \"network-metrics-daemon-g5nkq\" (UID: \"4c721b01-f754-4b83-87fe-94689fb629c9\") " pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:55:50.136631 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:50.136603    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:55:50.136747 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:50.136665    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs podName:4c721b01-f754-4b83-87fe-94689fb629c9 nodeName:}" failed. No retries permitted until 2026-09-11 12:55:52.136647215 +0000 UTC m=+6.076348541 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs") pod "network-metrics-daemon-g5nkq" (UID: "4c721b01-f754-4b83-87fe-94689fb629c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:55:50.137061 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:50.137042    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 12:55:50.137130 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:50.137096    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret podName:da6d1a19-06a1-4a7a-bbc2-593b438eeb41 nodeName:}" failed. No retries permitted until 2026-09-11 12:55:52.137081674 +0000 UTC m=+6.076782992 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret") pod "global-pull-secret-syncer-lbn4f" (UID: "da6d1a19-06a1-4a7a-bbc2-593b438eeb41") : object "kube-system"/"original-pull-secret" not registered
Sep 11 12:55:50.237579 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:50.236976    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9r54\" (UniqueName: \"kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54\") pod \"network-check-target-6k8nb\" (UID: \"0a1899d1-b90c-4b2f-b49c-06c3a2e3533d\") " pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:55:50.237579 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:50.237141    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 12:55:50.237579 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:50.237160    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 12:55:50.237579 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:50.237173    2582 projected.go:196] Error preparing data for projected volume kube-api-access-t9r54 for pod openshift-network-diagnostics/network-check-target-6k8nb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:55:50.237579 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:50.237227    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54 podName:0a1899d1-b90c-4b2f-b49c-06c3a2e3533d nodeName:}" failed. No retries permitted until 2026-09-11 12:55:52.23721127 +0000 UTC m=+6.176912595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9r54" (UniqueName: "kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54") pod "network-check-target-6k8nb" (UID: "0a1899d1-b90c-4b2f-b49c-06c3a2e3533d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:55:50.393537 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:50.391498    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 12:55:50.454546 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:50.454375    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 12:50:48 +0000 UTC" deadline="2028-02-28 02:31:53.520216374 +0000 UTC"
Sep 11 12:55:50.454546 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:50.454435    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12829h36m3.065786216s"
Sep 11 12:55:50.470467 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:50.469836    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:50.470467 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:50.469954    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:55:50.470467 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:50.470327    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:55:50.470467 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:50.470408    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:55:50.471181 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:50.470839    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:55:50.471181 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:50.470942    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:55:50.530634 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:50.530603    2582 generic.go:356] "Generic (PLEG): container finished" podID="b4c3351bb8d511ff78a7dff4fde75874" containerID="1fb62555d33dda306864c159a89d8483c2d4a09058a57b6aa543fb21b80208c2" exitCode=0
Sep 11 12:55:50.531460 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:50.531437    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" event={"ID":"b4c3351bb8d511ff78a7dff4fde75874","Type":"ContainerDied","Data":"1fb62555d33dda306864c159a89d8483c2d4a09058a57b6aa543fb21b80208c2"}
Sep 11 12:55:51.558480 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:51.558099    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" event={"ID":"b4c3351bb8d511ff78a7dff4fde75874","Type":"ContainerStarted","Data":"0079d67e86d61287c589bd45003d7a2a17d9a1bd58a234685e94276b6d36b91f"}
Sep 11 12:55:52.154286 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:52.154032    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:52.154471 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:52.154305    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs\") pod \"network-metrics-daemon-g5nkq\" (UID: \"4c721b01-f754-4b83-87fe-94689fb629c9\") " pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:55:52.154572 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:52.154557    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:55:52.154636 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:52.154629    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs podName:4c721b01-f754-4b83-87fe-94689fb629c9 nodeName:}" failed. No retries permitted until 2026-09-11 12:55:56.154610437 +0000 UTC m=+10.094311751 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs") pod "network-metrics-daemon-g5nkq" (UID: "4c721b01-f754-4b83-87fe-94689fb629c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:55:52.155029 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:52.155010    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 12:55:52.155137 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:52.155076    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret podName:da6d1a19-06a1-4a7a-bbc2-593b438eeb41 nodeName:}" failed. No retries permitted until 2026-09-11 12:55:56.155059506 +0000 UTC m=+10.094760832 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret") pod "global-pull-secret-syncer-lbn4f" (UID: "da6d1a19-06a1-4a7a-bbc2-593b438eeb41") : object "kube-system"/"original-pull-secret" not registered
Sep 11 12:55:52.254946 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:52.254875    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9r54\" (UniqueName: \"kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54\") pod \"network-check-target-6k8nb\" (UID: \"0a1899d1-b90c-4b2f-b49c-06c3a2e3533d\") " pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:55:52.255107 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:52.255020    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 12:55:52.255107 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:52.255043    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 12:55:52.255107 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:52.255055    2582 projected.go:196] Error preparing data for projected volume kube-api-access-t9r54 for pod openshift-network-diagnostics/network-check-target-6k8nb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:55:52.255107 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:52.255106    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54 podName:0a1899d1-b90c-4b2f-b49c-06c3a2e3533d nodeName:}" failed. No retries permitted until 2026-09-11 12:55:56.255089405 +0000 UTC m=+10.194790730 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9r54" (UniqueName: "kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54") pod "network-check-target-6k8nb" (UID: "0a1899d1-b90c-4b2f-b49c-06c3a2e3533d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:55:52.470573 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:52.470497    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:52.470738 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:52.470637    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:55:52.471035 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:52.471013    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:55:52.471143 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:52.471121    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:55:52.471276 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:52.471260    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:55:52.471434 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:52.471399    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:55:54.470272 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:54.469803    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:54.470272 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:54.469803    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:55:54.470272 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:54.469953    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:55:54.470272 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:54.469935    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:55:54.470272 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:54.470086    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:55:54.470272 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:54.470149    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:55:56.185134 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:56.185092    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:56.185625 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:56.185143    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs\") pod \"network-metrics-daemon-g5nkq\" (UID: \"4c721b01-f754-4b83-87fe-94689fb629c9\") " pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:55:56.185625 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:56.185254    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 12:55:56.185625 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:56.185332    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret podName:da6d1a19-06a1-4a7a-bbc2-593b438eeb41 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:04.185313695 +0000 UTC m=+18.125015022 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret") pod "global-pull-secret-syncer-lbn4f" (UID: "da6d1a19-06a1-4a7a-bbc2-593b438eeb41") : object "kube-system"/"original-pull-secret" not registered
Sep 11 12:55:56.185625 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:56.185257    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:55:56.185625 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:56.185431    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs podName:4c721b01-f754-4b83-87fe-94689fb629c9 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:04.185397777 +0000 UTC m=+18.125099105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs") pod "network-metrics-daemon-g5nkq" (UID: "4c721b01-f754-4b83-87fe-94689fb629c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:55:56.286462 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:56.286372    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9r54\" (UniqueName: \"kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54\") pod \"network-check-target-6k8nb\" (UID: \"0a1899d1-b90c-4b2f-b49c-06c3a2e3533d\") " pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:55:56.286619 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:56.286550    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 12:55:56.286619 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:56.286578    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 12:55:56.286619 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:56.286592    2582 projected.go:196] Error preparing data for projected volume kube-api-access-t9r54 for pod openshift-network-diagnostics/network-check-target-6k8nb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:55:56.286826 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:56.286658    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54 podName:0a1899d1-b90c-4b2f-b49c-06c3a2e3533d nodeName:}" failed. No retries permitted until 2026-09-11 12:56:04.286639083 +0000 UTC m=+18.226340407 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9r54" (UniqueName: "kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54") pod "network-check-target-6k8nb" (UID: "0a1899d1-b90c-4b2f-b49c-06c3a2e3533d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:55:56.473112 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:56.473084    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:55:56.473264 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:56.473191    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:55:56.473560 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:56.473540    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:56.473678 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:56.473656    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:55:56.473744 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:56.473712    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:55:56.473790 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:56.473773    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:55:58.470171 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:58.469614    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:55:58.470171 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:58.469748    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:55:58.470171 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:58.469557    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:55:58.470171 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:55:58.470135    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:55:58.470791 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:58.470243    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:55:58.470791 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:55:58.470356    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:56:00.470173 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:00.470137    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:00.470646 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:00.470135    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:00.470646 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:00.470241    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:56:00.470646 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:00.470135    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:00.470646 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:00.470343    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:56:00.470646 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:00.470456    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:56:02.472804 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:02.472771    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:02.472804 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:02.472789    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:02.472804 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:02.472798    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:02.473289 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:02.472907    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:56:02.473289 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:02.472974    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:56:02.473289 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:02.473064    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:56:04.252261 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:04.252222    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:04.252261 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:04.252270    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs\") pod \"network-metrics-daemon-g5nkq\" (UID: \"4c721b01-f754-4b83-87fe-94689fb629c9\") " pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:04.252801 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:04.252380    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 12:56:04.252801 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:04.252381    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:56:04.252801 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:04.252451    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret podName:da6d1a19-06a1-4a7a-bbc2-593b438eeb41 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:20.252434392 +0000 UTC m=+34.192135724 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret") pod "global-pull-secret-syncer-lbn4f" (UID: "da6d1a19-06a1-4a7a-bbc2-593b438eeb41") : object "kube-system"/"original-pull-secret" not registered
Sep 11 12:56:04.252801 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:04.252506    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs podName:4c721b01-f754-4b83-87fe-94689fb629c9 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:20.252485157 +0000 UTC m=+34.192186473 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs") pod "network-metrics-daemon-g5nkq" (UID: "4c721b01-f754-4b83-87fe-94689fb629c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:56:04.353476 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:04.353444    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9r54\" (UniqueName: \"kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54\") pod \"network-check-target-6k8nb\" (UID: \"0a1899d1-b90c-4b2f-b49c-06c3a2e3533d\") " pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:04.353652 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:04.353627    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 12:56:04.353720 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:04.353666    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 12:56:04.353720 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:04.353680    2582 projected.go:196] Error preparing data for projected volume kube-api-access-t9r54 for pod openshift-network-diagnostics/network-check-target-6k8nb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:56:04.353821 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:04.353742    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54 podName:0a1899d1-b90c-4b2f-b49c-06c3a2e3533d nodeName:}" failed. No retries permitted until 2026-09-11 12:56:20.353725434 +0000 UTC m=+34.293426765 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9r54" (UniqueName: "kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54") pod "network-check-target-6k8nb" (UID: "0a1899d1-b90c-4b2f-b49c-06c3a2e3533d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:56:04.469202 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:04.469173    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:04.469202 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:04.469188    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:04.469381 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:04.469217    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:04.469381 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:04.469322    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:56:04.469381 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:04.469348    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:56:04.469536 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:04.469433    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:56:06.470142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:06.470102    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:06.470621 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:06.470207    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:56:06.470691 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:06.470642    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:06.470774 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:06.470747    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:56:06.470830 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:06.470795    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:06.470935 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:06.470902    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:56:07.594163 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.593974    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8" event={"ID":"3fee639e-a752-4a3c-ba28-503b0dd550af","Type":"ContainerStarted","Data":"2c9b9f07cc1bd01e94d61009c88d38d90be86179cfa887890dcc775157b1c789"}
Sep 11 12:56:07.595468 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.595440    2582 generic.go:356] "Generic (PLEG): container finished" podID="63cdffd2-b869-4eea-b71a-5c2d91da8429" containerID="3daf4c123cc9761e5b81efced592b0b67ee8973b6d079915eefcf2c42c612721" exitCode=0
Sep 11 12:56:07.595551 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.595520    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9z6v" event={"ID":"63cdffd2-b869-4eea-b71a-5c2d91da8429","Type":"ContainerDied","Data":"3daf4c123cc9761e5b81efced592b0b67ee8973b6d079915eefcf2c42c612721"}
Sep 11 12:56:07.597079 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.597060    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sm29q" event={"ID":"b4af872d-253e-474d-ada0-07f8ac808485","Type":"ContainerStarted","Data":"71b05cf42e398661534ed9052a52a493307a9ab6b59435b08f038930cb119a76"}
Sep 11 12:56:07.598530 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.598497    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8lsb4" event={"ID":"8c08440e-84a0-40a3-b171-92220128ee4c","Type":"ContainerStarted","Data":"1c5ff0ea960f95c39ecb831dd30ae6b279f947b19294e14914fdcfde4c2c11c9"}
Sep 11 12:56:07.600041 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.600019    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tgmlr" event={"ID":"01226bcb-c885-4780-bf4a-7dfd31f4e919","Type":"ContainerStarted","Data":"60213981f907872087f761e84a877b795843686eacc8cb3e830426804fb22aa1"}
Sep 11 12:56:07.602819 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.602798    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj" event={"ID":"76e98787-2ece-4cfe-a0ef-57360c48e2a3","Type":"ContainerStarted","Data":"82a614cad6905b11fe96f5ad14ab9a06bde7b4a3955d6292fc20ae8f01b9947f"}
Sep 11 12:56:07.602918 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.602858    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj" event={"ID":"76e98787-2ece-4cfe-a0ef-57360c48e2a3","Type":"ContainerStarted","Data":"d9bad77a1b2754ae925f65e2c18e59b647197ab0f34a1b5ad3c5c365e3965212"}
Sep 11 12:56:07.602918 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.602895    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj" event={"ID":"76e98787-2ece-4cfe-a0ef-57360c48e2a3","Type":"ContainerStarted","Data":"3da67a05eb561f0f3b56f535fb73be4a5cf4fb9f3045c32822ef5337356696ad"}
Sep 11 12:56:07.603010 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.602930    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj" event={"ID":"76e98787-2ece-4cfe-a0ef-57360c48e2a3","Type":"ContainerStarted","Data":"6a910129e923f064baf0bac78ec59854bf12616d1f0217b3024b6c456e5d0e5d"}
Sep 11 12:56:07.604086 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.604063    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s49mh" event={"ID":"8f2bd220-7fb1-4c41-bbc4-28f57fdf9a27","Type":"ContainerStarted","Data":"b23208dfd4869754d725bc8d17da781163cb4a77012e8bc5deea25ccfe955ad9"}
Sep 11 12:56:07.605368 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.605350    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zkjt7" event={"ID":"b3d2a78d-a895-4b2f-b965-bd1cb39bdff2","Type":"ContainerStarted","Data":"6799cb14ef114094a2abf08def15289e575def7e1a0b76799f7478df80acb1d0"}
Sep 11 12:56:07.617855 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.617818    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" podStartSLOduration=19.617807608 podStartE2EDuration="19.617807608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:55:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:55:51.575487267 +0000 UTC m=+5.515188601" watchObservedRunningTime="2026-09-11 12:56:07.617807608 +0000 UTC m=+21.557508941"
Sep 11 12:56:07.637556 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.637517    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-s49mh" podStartSLOduration=3.861952061 podStartE2EDuration="21.637507782s" totalImagesPullingTime="17.775555721s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:07.636959471 +0000 UTC m=+21.576660803" watchObservedRunningTime="2026-09-11 12:56:07.637507782 +0000 UTC m=+21.577209152"
Sep 11 12:56:07.652279 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.652236    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8lsb4" podStartSLOduration=3.88369489 podStartE2EDuration="21.65222126s" totalImagesPullingTime="17.76852637s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:07.65186939 +0000 UTC m=+21.591570746" watchObservedRunningTime="2026-09-11 12:56:07.65222126 +0000 UTC m=+21.591922595"
Sep 11 12:56:07.673791 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.673757    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zkjt7" podStartSLOduration=11.413303296 podStartE2EDuration="21.67374593s" totalImagesPullingTime="10.260442634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:07.673109668 +0000 UTC m=+21.612811001" watchObservedRunningTime="2026-09-11 12:56:07.67374593 +0000 UTC m=+21.613447263"
Sep 11 12:56:07.690443 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.690215    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tgmlr" podStartSLOduration=3.79869424 podStartE2EDuration="21.690199761s" totalImagesPullingTime="17.891505521s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:07.689415385 +0000 UTC m=+21.629116719" watchObservedRunningTime="2026-09-11 12:56:07.690199761 +0000 UTC m=+21.629901098"
Sep 11 12:56:07.704955 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:07.704924    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sm29q" podStartSLOduration=3.926936144 podStartE2EDuration="21.704912746s" totalImagesPullingTime="17.777976602s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:07.704551189 +0000 UTC m=+21.644252522" watchObservedRunningTime="2026-09-11 12:56:07.704912746 +0000 UTC m=+21.644614132"
Sep 11 12:56:08.299164 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:08.299140    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 12:56:08.470182 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:08.470121    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:08.470294 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:08.470121    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:08.470294 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:08.470240    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:56:08.470294 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:08.470125    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:08.470456 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:08.470295    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:56:08.470456 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:08.470381    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:56:08.481943 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:08.481864    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T12:56:08.299160613Z","UUID":"a8f01ff8-4e7b-455c-ba61-5bcfe63ad645","Handler":null,"Name":"","Endpoint":""}
Sep 11 12:56:08.483352 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:08.483333    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 12:56:08.483463 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:08.483365    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 12:56:08.610542 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:08.610510    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj" event={"ID":"76e98787-2ece-4cfe-a0ef-57360c48e2a3","Type":"ContainerStarted","Data":"56770834bc0a9ba46a2f5e10c30656fd6041ed2e89f5727a3575ecc09cd07982"}
Sep 11 12:56:08.610870 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:08.610594    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj" event={"ID":"76e98787-2ece-4cfe-a0ef-57360c48e2a3","Type":"ContainerStarted","Data":"216e04fb545fcdbb597e21df307366e6580e4c17653e06065e01dd9d2aa63853"}
Sep 11 12:56:08.612303 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:08.612281    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bpmlr" event={"ID":"eee29df6-f28a-485c-9e0b-e53a3b6d039e","Type":"ContainerStarted","Data":"8ec6cc780dfd0fb8495d2ecaf25f267334e1228697d2849036911f85017c854d"}
Sep 11 12:56:08.614334 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:08.614300    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8" event={"ID":"3fee639e-a752-4a3c-ba28-503b0dd550af","Type":"ContainerStarted","Data":"68cee73865428261c020375c34fa4108458c8f8c06c3272765e432d07989f30b"}
Sep 11 12:56:08.627559 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:08.627521    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bpmlr" podStartSLOduration=4.853355615 podStartE2EDuration="22.627510218s" totalImagesPullingTime="17.774154603s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:08.627188915 +0000 UTC m=+22.566890249" watchObservedRunningTime="2026-09-11 12:56:08.627510218 +0000 UTC m=+22.567211582"
Sep 11 12:56:09.619407 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:09.619370    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8" event={"ID":"3fee639e-a752-4a3c-ba28-503b0dd550af","Type":"ContainerStarted","Data":"cec9cfaba6d4720435c0ce9c60efe662e05289cc7a722a6fce9d3c02561176e2"}
Sep 11 12:56:10.470191 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:10.470158    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:10.470446 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:10.470158    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:10.470446 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:10.470265    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:56:10.470446 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:10.470296    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:10.470446 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:10.470386    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:56:10.470637 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:10.470509    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:56:10.625784 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:10.625746    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj" event={"ID":"76e98787-2ece-4cfe-a0ef-57360c48e2a3","Type":"ContainerStarted","Data":"48441b7d41304fcee6b5bd383cd0411f42c0ff1e523d74c20774a43f504dbdd3"}
Sep 11 12:56:11.171310 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:11.171272    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zkjt7"
Sep 11 12:56:11.171875 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:11.171850    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zkjt7"
Sep 11 12:56:11.188811 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:11.188767    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qbvd8" podStartSLOduration=5.813770979 podStartE2EDuration="25.188749629s" totalImagesPullingTime="19.37497865s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:55:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:09.637896966 +0000 UTC m=+23.577598300" watchObservedRunningTime="2026-09-11 12:56:11.188749629 +0000 UTC m=+25.128450966"
Sep 11 12:56:12.469981 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:12.469817    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:12.470586 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:12.469880    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:12.470586 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:12.470056    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:56:12.470586 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:12.470170    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:56:12.470586 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:12.469911    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:12.470586 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:12.470273    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:56:12.630941 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:12.630914    2582 generic.go:356] "Generic (PLEG): container finished" podID="63cdffd2-b869-4eea-b71a-5c2d91da8429" containerID="16e0f13aceda617c4fd26e599af3975f803b90a68dd9026884c44cdea0ed5834" exitCode=0
Sep 11 12:56:12.631054 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:12.631003    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9z6v" event={"ID":"63cdffd2-b869-4eea-b71a-5c2d91da8429","Type":"ContainerDied","Data":"16e0f13aceda617c4fd26e599af3975f803b90a68dd9026884c44cdea0ed5834"}
Sep 11 12:56:12.634938 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:12.634919    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj" event={"ID":"76e98787-2ece-4cfe-a0ef-57360c48e2a3","Type":"ContainerStarted","Data":"93e4913dded1189810ae101ecc7ec7117a53705675ec95d66d9489008de08535"}
Sep 11 12:56:12.635186 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:12.635167    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:56:12.635293 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:12.635241    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:56:12.635351 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:12.635293    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:56:12.652548 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:12.650369    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:56:12.652548 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:12.650686    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:56:12.679055 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:12.679018    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj" podStartSLOduration=8.882924138 podStartE2EDuration="26.679007193s" totalImagesPullingTime="17.796083055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:12.67746653 +0000 UTC m=+26.617167864" watchObservedRunningTime="2026-09-11 12:56:12.679007193 +0000 UTC m=+26.618708527"
Sep 11 12:56:14.027209 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:14.027048    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lbn4f"]
Sep 11 12:56:14.027570 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:14.027295    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:14.027570 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:14.027378    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:56:14.027699 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:14.027679    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g5nkq"]
Sep 11 12:56:14.027783 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:14.027772    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:14.027898 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:14.027881    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:56:14.030082 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:14.030059    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6k8nb"]
Sep 11 12:56:14.030197 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:14.030162    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:14.030265 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:14.030244    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:56:14.642872 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:14.642839    2582 generic.go:356] "Generic (PLEG): container finished" podID="63cdffd2-b869-4eea-b71a-5c2d91da8429" containerID="9baa5c97d595f6d3dede12cdc30e14ab1719b21e8a7a03a16138006de4e5691b" exitCode=0
Sep 11 12:56:14.643127 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:14.642930    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9z6v" event={"ID":"63cdffd2-b869-4eea-b71a-5c2d91da8429","Type":"ContainerDied","Data":"9baa5c97d595f6d3dede12cdc30e14ab1719b21e8a7a03a16138006de4e5691b"}
Sep 11 12:56:15.148246 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:15.148215    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zkjt7"
Sep 11 12:56:15.148600 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:15.148440    2582 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 12:56:15.148905 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:15.148876    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zkjt7"
Sep 11 12:56:15.469382 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:15.469313    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:15.469511 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:15.469468    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:56:15.647692 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:15.647665    2582 generic.go:356] "Generic (PLEG): container finished" podID="63cdffd2-b869-4eea-b71a-5c2d91da8429" containerID="6c489162856f8cc648e0479b69d745cb251405471158b057e211bfc7ca790e1f" exitCode=0
Sep 11 12:56:15.647823 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:15.647763    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9z6v" event={"ID":"63cdffd2-b869-4eea-b71a-5c2d91da8429","Type":"ContainerDied","Data":"6c489162856f8cc648e0479b69d745cb251405471158b057e211bfc7ca790e1f"}
Sep 11 12:56:16.471098 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:16.471067    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:16.471526 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:16.471176    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:56:16.471526 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:16.471262    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:16.471526 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:16.471393    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:56:17.470312 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:17.470131    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:17.470493 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:17.470381    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:56:18.470148 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:18.470072    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:18.470574 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:18.470201    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8nb" podUID="0a1899d1-b90c-4b2f-b49c-06c3a2e3533d"
Sep 11 12:56:18.470574 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:18.470260    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:18.470574 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:18.470405    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g5nkq" podUID="4c721b01-f754-4b83-87fe-94689fb629c9"
Sep 11 12:56:19.469390 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:19.469361    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:19.469660 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:19.469490    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbn4f" podUID="da6d1a19-06a1-4a7a-bbc2-593b438eeb41"
Sep 11 12:56:19.906467 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:19.906442    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeReady"
Sep 11 12:56:19.906826 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:19.906546    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 12:56:19.976535 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:19.976509    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nt9jt"]
Sep 11 12:56:19.993526 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:19.993507    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vf4j6"]
Sep 11 12:56:19.993656 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:19.993641    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:19.996452 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:19.996412    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 12:56:19.996576 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:19.996474    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l85nk\""
Sep 11 12:56:19.996898 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:19.996879    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 12:56:20.012006 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.011988    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nt9jt"]
Sep 11 12:56:20.012094 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.012018    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vf4j6"]
Sep 11 12:56:20.012094 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.012078    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vf4j6"
Sep 11 12:56:20.015303 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.015286    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pwbps\""
Sep 11 12:56:20.015596 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.015578    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 12:56:20.015668 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.015602    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 12:56:20.015723 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.015607    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 12:56:20.175702 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.175650    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d-cert\") pod \"ingress-canary-vf4j6\" (UID: \"c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d\") " pod="openshift-ingress-canary/ingress-canary-vf4j6"
Sep 11 12:56:20.175702 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.175677    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60ebfb6b-c42a-4634-ad12-24056202391e-config-volume\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:20.175702 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.175700    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60ebfb6b-c42a-4634-ad12-24056202391e-tmp-dir\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:20.175876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.175756    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:20.175876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.175788    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tf6m\" (UniqueName: \"kubernetes.io/projected/60ebfb6b-c42a-4634-ad12-24056202391e-kube-api-access-7tf6m\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:20.175876 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.175823    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrhpf\" (UniqueName: \"kubernetes.io/projected/c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d-kube-api-access-mrhpf\") pod \"ingress-canary-vf4j6\" (UID: \"c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d\") " pod="openshift-ingress-canary/ingress-canary-vf4j6"
Sep 11 12:56:20.276084 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.276059    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d-cert\") pod \"ingress-canary-vf4j6\" (UID: \"c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d\") " pod="openshift-ingress-canary/ingress-canary-vf4j6"
Sep 11 12:56:20.276193 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.276091    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60ebfb6b-c42a-4634-ad12-24056202391e-config-volume\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:20.276193 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.276118    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60ebfb6b-c42a-4634-ad12-24056202391e-tmp-dir\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:20.276193 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.276136    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:20.276346 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:20.276214    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 12:56:20.276346 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.276253    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs\") pod \"network-metrics-daemon-g5nkq\" (UID: \"4c721b01-f754-4b83-87fe-94689fb629c9\") " pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:20.276346 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:20.276263    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret podName:da6d1a19-06a1-4a7a-bbc2-593b438eeb41 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:52.27624683 +0000 UTC m=+66.215948153 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret") pod "global-pull-secret-syncer-lbn4f" (UID: "da6d1a19-06a1-4a7a-bbc2-593b438eeb41") : object "kube-system"/"original-pull-secret" not registered
Sep 11 12:56:20.276346 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.276306    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:20.276346 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.276334    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tf6m\" (UniqueName: \"kubernetes.io/projected/60ebfb6b-c42a-4634-ad12-24056202391e-kube-api-access-7tf6m\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:20.276620 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.276364    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrhpf\" (UniqueName: \"kubernetes.io/projected/c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d-kube-api-access-mrhpf\") pod \"ingress-canary-vf4j6\" (UID: \"c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d\") " pod="openshift-ingress-canary/ingress-canary-vf4j6"
Sep 11 12:56:20.276620 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.276457    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60ebfb6b-c42a-4634-ad12-24056202391e-tmp-dir\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:20.276620 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:20.276548    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 12:56:20.276620 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:20.276589    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls podName:60ebfb6b-c42a-4634-ad12-24056202391e nodeName:}" failed. No retries permitted until 2026-09-11 12:56:20.776574621 +0000 UTC m=+34.716275931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls") pod "dns-default-nt9jt" (UID: "60ebfb6b-c42a-4634-ad12-24056202391e") : secret "dns-default-metrics-tls" not found
Sep 11 12:56:20.276620 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:20.276602    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:56:20.276858 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:20.276649    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs podName:4c721b01-f754-4b83-87fe-94689fb629c9 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:52.276633975 +0000 UTC m=+66.216335288 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs") pod "network-metrics-daemon-g5nkq" (UID: "4c721b01-f754-4b83-87fe-94689fb629c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 12:56:20.276858 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.276768    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60ebfb6b-c42a-4634-ad12-24056202391e-config-volume\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:20.281138 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.281113    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d-cert\") pod \"ingress-canary-vf4j6\" (UID: \"c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d\") " pod="openshift-ingress-canary/ingress-canary-vf4j6"
Sep 11 12:56:20.284623 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.284596    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tf6m\" (UniqueName: \"kubernetes.io/projected/60ebfb6b-c42a-4634-ad12-24056202391e-kube-api-access-7tf6m\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:20.287603 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.287581    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrhpf\" (UniqueName: \"kubernetes.io/projected/c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d-kube-api-access-mrhpf\") pod \"ingress-canary-vf4j6\" (UID: \"c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d\") " pod="openshift-ingress-canary/ingress-canary-vf4j6"
Sep 11 12:56:20.322478 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.322456    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vf4j6"
Sep 11 12:56:20.377680 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.377657    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9r54\" (UniqueName: \"kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54\") pod \"network-check-target-6k8nb\" (UID: \"0a1899d1-b90c-4b2f-b49c-06c3a2e3533d\") " pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:20.377790 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:20.377778    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 12:56:20.377851 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:20.377797    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 12:56:20.377851 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:20.377806    2582 projected.go:196] Error preparing data for projected volume kube-api-access-t9r54 for pod openshift-network-diagnostics/network-check-target-6k8nb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:56:20.377851 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:20.377849    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54 podName:0a1899d1-b90c-4b2f-b49c-06c3a2e3533d nodeName:}" failed. No retries permitted until 2026-09-11 12:56:52.377836152 +0000 UTC m=+66.317537475 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9r54" (UniqueName: "kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54") pod "network-check-target-6k8nb" (UID: "0a1899d1-b90c-4b2f-b49c-06c3a2e3533d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 12:56:20.470102 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.470076    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:20.470293 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.470268    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:20.473542 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.473522    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 12:56:20.474308 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.474292    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 12:56:20.474560 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.474545    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7r4pp\""
Sep 11 12:56:20.474611 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.474571    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 12:56:20.474676 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.474551    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w7fbx\""
Sep 11 12:56:20.652514 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.652473    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vf4j6"]
Sep 11 12:56:20.656014 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:56:20.655992    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc96a7bc4_bd26_4324_b364_2bc0b5f2ea9d.slice/crio-0bcdca955e80f170f92c8628a29fd79dbb12654a119b7ef7184308fae6934667 WatchSource:0}: Error finding container 0bcdca955e80f170f92c8628a29fd79dbb12654a119b7ef7184308fae6934667: Status 404 returned error can't find the container with id 0bcdca955e80f170f92c8628a29fd79dbb12654a119b7ef7184308fae6934667
Sep 11 12:56:20.661949 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.661920    2582 generic.go:356] "Generic (PLEG): container finished" podID="63cdffd2-b869-4eea-b71a-5c2d91da8429" containerID="3351adf7019172609f06e77d16d4f09c6fccdb490d419b8f45d4c12c7b11e51d" exitCode=0
Sep 11 12:56:20.662051 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.662002    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9z6v" event={"ID":"63cdffd2-b869-4eea-b71a-5c2d91da8429","Type":"ContainerDied","Data":"3351adf7019172609f06e77d16d4f09c6fccdb490d419b8f45d4c12c7b11e51d"}
Sep 11 12:56:20.663157 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.663131    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vf4j6" event={"ID":"c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d","Type":"ContainerStarted","Data":"0bcdca955e80f170f92c8628a29fd79dbb12654a119b7ef7184308fae6934667"}
Sep 11 12:56:20.781375 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:20.781353    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:20.781512 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:20.781492    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 12:56:20.781578 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:20.781554    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls podName:60ebfb6b-c42a-4634-ad12-24056202391e nodeName:}" failed. No retries permitted until 2026-09-11 12:56:21.781536001 +0000 UTC m=+35.721237337 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls") pod "dns-default-nt9jt" (UID: "60ebfb6b-c42a-4634-ad12-24056202391e") : secret "dns-default-metrics-tls" not found
Sep 11 12:56:21.469781 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:21.469752    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:21.472705 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:21.472686    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 12:56:21.669443 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:21.669359    2582 generic.go:356] "Generic (PLEG): container finished" podID="63cdffd2-b869-4eea-b71a-5c2d91da8429" containerID="ef659946698eeeaabe930533fea3a07e6492704e4b08e49efa958e5a26368dfc" exitCode=0
Sep 11 12:56:21.669443 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:21.669405    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9z6v" event={"ID":"63cdffd2-b869-4eea-b71a-5c2d91da8429","Type":"ContainerDied","Data":"ef659946698eeeaabe930533fea3a07e6492704e4b08e49efa958e5a26368dfc"}
Sep 11 12:56:21.789479 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:21.789408    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:21.789977 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:21.789669    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 12:56:21.789977 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:21.789791    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls podName:60ebfb6b-c42a-4634-ad12-24056202391e nodeName:}" failed. No retries permitted until 2026-09-11 12:56:23.789771708 +0000 UTC m=+37.729473042 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls") pod "dns-default-nt9jt" (UID: "60ebfb6b-c42a-4634-ad12-24056202391e") : secret "dns-default-metrics-tls" not found
Sep 11 12:56:22.319917 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.319889    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-7tf5q"]
Sep 11 12:56:22.351590 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.351563    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-7tf5q"]
Sep 11 12:56:22.351740 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.351661    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7tf5q"
Sep 11 12:56:22.354460 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.354439    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zr5d5\""
Sep 11 12:56:22.354633 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.354589    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 11 12:56:22.355559 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.355536    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 11 12:56:22.392961 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.392939    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgvmt\" (UniqueName: \"kubernetes.io/projected/44840c25-e5a6-4882-9316-81ecf943805a-kube-api-access-hgvmt\") pod \"migrator-8b4d697c5-7tf5q\" (UID: \"44840c25-e5a6-4882-9316-81ecf943805a\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7tf5q"
Sep 11 12:56:22.493256 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.493223    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgvmt\" (UniqueName: \"kubernetes.io/projected/44840c25-e5a6-4882-9316-81ecf943805a-kube-api-access-hgvmt\") pod \"migrator-8b4d697c5-7tf5q\" (UID: \"44840c25-e5a6-4882-9316-81ecf943805a\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7tf5q"
Sep 11 12:56:22.503821 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.503793    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgvmt\" (UniqueName: \"kubernetes.io/projected/44840c25-e5a6-4882-9316-81ecf943805a-kube-api-access-hgvmt\") pod \"migrator-8b4d697c5-7tf5q\" (UID: \"44840c25-e5a6-4882-9316-81ecf943805a\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7tf5q"
Sep 11 12:56:22.661122 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.661048    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7tf5q"
Sep 11 12:56:22.675988 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.675963    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d9z6v" event={"ID":"63cdffd2-b869-4eea-b71a-5c2d91da8429","Type":"ContainerStarted","Data":"e9b52c1eabb1a6c897128ff139bcc71c1ba5dd8ed708aad97e3d3a4e3b901c80"}
Sep 11 12:56:22.696775 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.696731    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-d9z6v" podStartSLOduration=8.132843991 podStartE2EDuration="36.696715058s" totalImagesPullingTime="28.563871067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:55:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:22.69539735 +0000 UTC m=+36.635098683" watchObservedRunningTime="2026-09-11 12:56:22.696715058 +0000 UTC m=+36.636416392"
Sep 11 12:56:22.906640 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.906610    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fvrvd"]
Sep 11 12:56:22.925516 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.925461    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fvrvd"]
Sep 11 12:56:22.925614 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.925577    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:22.928323 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.928303    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-22vc9\""
Sep 11 12:56:22.928443 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.928333    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 12:56:22.928596 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.928575    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 12:56:22.928698 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.928575    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 12:56:22.928698 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.928649    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 12:56:22.996745 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.996716    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f84wl\" (UniqueName: \"kubernetes.io/projected/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-api-access-f84wl\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:22.996875 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.996763    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0299ca51-7147-46ea-83f9-90e9efca03c7-data-volume\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:22.996875 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.996793    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:22.996957 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.996903    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:22.996957 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:22.996943    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0299ca51-7147-46ea-83f9-90e9efca03c7-crio-socket\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:23.097671 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:23.097648    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f84wl\" (UniqueName: \"kubernetes.io/projected/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-api-access-f84wl\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:23.097785 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:23.097687    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0299ca51-7147-46ea-83f9-90e9efca03c7-data-volume\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:23.097785 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:23.097704    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:23.097785 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:23.097751    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:23.097785 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:23.097769    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0299ca51-7147-46ea-83f9-90e9efca03c7-crio-socket\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:23.097939 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:23.097886    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 12:56:23.097939 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:23.097894    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 12:56:23.098025 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:23.097946    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0299ca51-7147-46ea-83f9-90e9efca03c7-crio-socket\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:23.098025 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:23.097969    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm podName:0299ca51-7147-46ea-83f9-90e9efca03c7 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:23.597949795 +0000 UTC m=+37.537651123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fvrvd" (UID: "0299ca51-7147-46ea-83f9-90e9efca03c7") : configmap "kube-rbac-proxy" not found
Sep 11 12:56:23.098025 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:23.097993    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls podName:0299ca51-7147-46ea-83f9-90e9efca03c7 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:23.597978288 +0000 UTC m=+37.537679600 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fvrvd" (UID: "0299ca51-7147-46ea-83f9-90e9efca03c7") : secret "insights-runtime-extractor-tls" not found
Sep 11 12:56:23.098221 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:23.098201    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0299ca51-7147-46ea-83f9-90e9efca03c7-data-volume\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:23.110025 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:23.110005    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f84wl\" (UniqueName: \"kubernetes.io/projected/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-api-access-f84wl\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:23.225194 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:23.225174    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-7tf5q"]
Sep 11 12:56:23.229989 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:56:23.229938    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44840c25_e5a6_4882_9316_81ecf943805a.slice/crio-10a119d99f552bfbdbf90572e4bf3e157661780ed734c6d70134b9e7f60f4eac WatchSource:0}: Error finding container 10a119d99f552bfbdbf90572e4bf3e157661780ed734c6d70134b9e7f60f4eac: Status 404 returned error can't find the container with id 10a119d99f552bfbdbf90572e4bf3e157661780ed734c6d70134b9e7f60f4eac
Sep 11 12:56:23.601064 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:23.601043    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:23.601587 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:23.601095    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:23.601587 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:23.601157    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 12:56:23.601587 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:23.601189    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 12:56:23.601587 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:23.601208    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm podName:0299ca51-7147-46ea-83f9-90e9efca03c7 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:24.601194764 +0000 UTC m=+38.540896076 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fvrvd" (UID: "0299ca51-7147-46ea-83f9-90e9efca03c7") : configmap "kube-rbac-proxy" not found
Sep 11 12:56:23.601587 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:23.601240    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls podName:0299ca51-7147-46ea-83f9-90e9efca03c7 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:24.601227256 +0000 UTC m=+38.540928566 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fvrvd" (UID: "0299ca51-7147-46ea-83f9-90e9efca03c7") : secret "insights-runtime-extractor-tls" not found
Sep 11 12:56:23.679875 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:23.679854    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vf4j6" event={"ID":"c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d","Type":"ContainerStarted","Data":"8fcedd38643c9b4bb93462d7dcd5879d3c011e844015d5a22458b2bf8683d1f4"}
Sep 11 12:56:23.680966 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:23.680943    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7tf5q" event={"ID":"44840c25-e5a6-4882-9316-81ecf943805a","Type":"ContainerStarted","Data":"10a119d99f552bfbdbf90572e4bf3e157661780ed734c6d70134b9e7f60f4eac"}
Sep 11 12:56:23.696870 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:23.696834    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vf4j6" podStartSLOduration=2.140757425 podStartE2EDuration="4.696821286s" totalImagesPullingTime="2.556063861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:23.696380852 +0000 UTC m=+37.636082187" watchObservedRunningTime="2026-09-11 12:56:23.696821286 +0000 UTC m=+37.636522610"
Sep 11 12:56:23.802207 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:23.802173    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:23.802333 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:23.802305    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 12:56:23.802387 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:23.802373    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls podName:60ebfb6b-c42a-4634-ad12-24056202391e nodeName:}" failed. No retries permitted until 2026-09-11 12:56:27.80235477 +0000 UTC m=+41.742056082 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls") pod "dns-default-nt9jt" (UID: "60ebfb6b-c42a-4634-ad12-24056202391e") : secret "dns-default-metrics-tls" not found
Sep 11 12:56:24.611917 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:24.611880    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:24.612371 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:24.611938    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:24.612371 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:24.612026    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 12:56:24.612371 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:24.612040    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 12:56:24.612371 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:24.612078    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm podName:0299ca51-7147-46ea-83f9-90e9efca03c7 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:26.612065 +0000 UTC m=+40.551766311 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fvrvd" (UID: "0299ca51-7147-46ea-83f9-90e9efca03c7") : configmap "kube-rbac-proxy" not found
Sep 11 12:56:24.612371 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:24.612106    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls podName:0299ca51-7147-46ea-83f9-90e9efca03c7 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:26.612092927 +0000 UTC m=+40.551794238 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fvrvd" (UID: "0299ca51-7147-46ea-83f9-90e9efca03c7") : secret "insights-runtime-extractor-tls" not found
Sep 11 12:56:25.087076 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.087049    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-ppzv7"]
Sep 11 12:56:25.099904 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.099881    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-ppzv7"]
Sep 11 12:56:25.100008 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.099971    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-ppzv7"
Sep 11 12:56:25.102709 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.102687    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 11 12:56:25.103700 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.103685    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 11 12:56:25.104076 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.103963    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 11 12:56:25.104076 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.103985    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 11 12:56:25.104076 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.104000    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-dfzzj\""
Sep 11 12:56:25.115853 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.115688    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f71fce0e-b4a2-4c1e-8442-a80756ebfc76-signing-key\") pod \"service-ca-568869fdc5-ppzv7\" (UID: \"f71fce0e-b4a2-4c1e-8442-a80756ebfc76\") " pod="openshift-service-ca/service-ca-568869fdc5-ppzv7"
Sep 11 12:56:25.115853 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.115839    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7d2z\" (UniqueName: \"kubernetes.io/projected/f71fce0e-b4a2-4c1e-8442-a80756ebfc76-kube-api-access-j7d2z\") pod \"service-ca-568869fdc5-ppzv7\" (UID: \"f71fce0e-b4a2-4c1e-8442-a80756ebfc76\") " pod="openshift-service-ca/service-ca-568869fdc5-ppzv7"
Sep 11 12:56:25.116001 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.115911    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f71fce0e-b4a2-4c1e-8442-a80756ebfc76-signing-cabundle\") pod \"service-ca-568869fdc5-ppzv7\" (UID: \"f71fce0e-b4a2-4c1e-8442-a80756ebfc76\") " pod="openshift-service-ca/service-ca-568869fdc5-ppzv7"
Sep 11 12:56:25.216995 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.216971    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f71fce0e-b4a2-4c1e-8442-a80756ebfc76-signing-cabundle\") pod \"service-ca-568869fdc5-ppzv7\" (UID: \"f71fce0e-b4a2-4c1e-8442-a80756ebfc76\") " pod="openshift-service-ca/service-ca-568869fdc5-ppzv7"
Sep 11 12:56:25.217088 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.217067    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f71fce0e-b4a2-4c1e-8442-a80756ebfc76-signing-key\") pod \"service-ca-568869fdc5-ppzv7\" (UID: \"f71fce0e-b4a2-4c1e-8442-a80756ebfc76\") " pod="openshift-service-ca/service-ca-568869fdc5-ppzv7"
Sep 11 12:56:25.217142 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.217091    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7d2z\" (UniqueName: \"kubernetes.io/projected/f71fce0e-b4a2-4c1e-8442-a80756ebfc76-kube-api-access-j7d2z\") pod \"service-ca-568869fdc5-ppzv7\" (UID: \"f71fce0e-b4a2-4c1e-8442-a80756ebfc76\") " pod="openshift-service-ca/service-ca-568869fdc5-ppzv7"
Sep 11 12:56:25.217624 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.217599    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f71fce0e-b4a2-4c1e-8442-a80756ebfc76-signing-cabundle\") pod \"service-ca-568869fdc5-ppzv7\" (UID: \"f71fce0e-b4a2-4c1e-8442-a80756ebfc76\") " pod="openshift-service-ca/service-ca-568869fdc5-ppzv7"
Sep 11 12:56:25.219607 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.219588    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f71fce0e-b4a2-4c1e-8442-a80756ebfc76-signing-key\") pod \"service-ca-568869fdc5-ppzv7\" (UID: \"f71fce0e-b4a2-4c1e-8442-a80756ebfc76\") " pod="openshift-service-ca/service-ca-568869fdc5-ppzv7"
Sep 11 12:56:25.225674 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.225648    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7d2z\" (UniqueName: \"kubernetes.io/projected/f71fce0e-b4a2-4c1e-8442-a80756ebfc76-kube-api-access-j7d2z\") pod \"service-ca-568869fdc5-ppzv7\" (UID: \"f71fce0e-b4a2-4c1e-8442-a80756ebfc76\") " pod="openshift-service-ca/service-ca-568869fdc5-ppzv7"
Sep 11 12:56:25.408869 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.408788    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-ppzv7"
Sep 11 12:56:25.455291 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.455269    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8lsb4_8c08440e-84a0-40a3-b171-92220128ee4c/dns-node-resolver/0.log"
Sep 11 12:56:25.686817 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.686745    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7tf5q" event={"ID":"44840c25-e5a6-4882-9316-81ecf943805a","Type":"ContainerStarted","Data":"2adf809d6816581aa87c8b3e716bdb4faf626986937708ec5f290d0843d3cbde"}
Sep 11 12:56:25.686817 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.686796    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7tf5q" event={"ID":"44840c25-e5a6-4882-9316-81ecf943805a","Type":"ContainerStarted","Data":"76d1942471e4a575c9600dc8ee0986338d842db1e2cb8aa24fa7efd161fe2909"}
Sep 11 12:56:25.697732 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.697705    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-ppzv7"]
Sep 11 12:56:25.701156 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:56:25.701135    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf71fce0e_b4a2_4c1e_8442_a80756ebfc76.slice/crio-580b814ca235e1142e6a02d056df7000097478ae06d2a0c5b972c190af463154 WatchSource:0}: Error finding container 580b814ca235e1142e6a02d056df7000097478ae06d2a0c5b972c190af463154: Status 404 returned error can't find the container with id 580b814ca235e1142e6a02d056df7000097478ae06d2a0c5b972c190af463154
Sep 11 12:56:25.722534 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:25.722490    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7tf5q" podStartSLOduration=2.212941149 podStartE2EDuration="3.72247331s" totalImagesPullingTime="1.509532161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:25.721512367 +0000 UTC m=+39.661213701" watchObservedRunningTime="2026-09-11 12:56:25.72247331 +0000 UTC m=+39.662174642"
Sep 11 12:56:26.626269 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:26.626238    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:26.626491 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:26.626302    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:26.626491 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:26.626374    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 12:56:26.626491 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:26.626396    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 12:56:26.626491 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:26.626446    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm podName:0299ca51-7147-46ea-83f9-90e9efca03c7 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:30.626416486 +0000 UTC m=+44.566132460 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fvrvd" (UID: "0299ca51-7147-46ea-83f9-90e9efca03c7") : configmap "kube-rbac-proxy" not found
Sep 11 12:56:26.626491 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:26.626485    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls podName:0299ca51-7147-46ea-83f9-90e9efca03c7 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:30.626465692 +0000 UTC m=+44.566167017 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fvrvd" (UID: "0299ca51-7147-46ea-83f9-90e9efca03c7") : secret "insights-runtime-extractor-tls" not found
Sep 11 12:56:26.654072 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:26.654041    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sm29q_b4af872d-253e-474d-ada0-07f8ac808485/node-ca/0.log"
Sep 11 12:56:26.689864 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:26.689835    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-ppzv7" event={"ID":"f71fce0e-b4a2-4c1e-8442-a80756ebfc76","Type":"ContainerStarted","Data":"580b814ca235e1142e6a02d056df7000097478ae06d2a0c5b972c190af463154"}
Sep 11 12:56:27.653361 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:27.653335    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vf4j6_c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d/serve-healthcheck-canary/0.log"
Sep 11 12:56:27.838047 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:27.838017    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:27.838367 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:27.838127    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 12:56:27.838367 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:27.838177    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls podName:60ebfb6b-c42a-4634-ad12-24056202391e nodeName:}" failed. No retries permitted until 2026-09-11 12:56:35.838166003 +0000 UTC m=+49.777867313 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls") pod "dns-default-nt9jt" (UID: "60ebfb6b-c42a-4634-ad12-24056202391e") : secret "dns-default-metrics-tls" not found
Sep 11 12:56:27.853713 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:27.853692    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-7tf5q_44840c25-e5a6-4882-9316-81ecf943805a/migrator/0.log"
Sep 11 12:56:28.050749 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:28.050725    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-7tf5q_44840c25-e5a6-4882-9316-81ecf943805a/graceful-termination/0.log"
Sep 11 12:56:28.695697 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:28.695663    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-ppzv7" event={"ID":"f71fce0e-b4a2-4c1e-8442-a80756ebfc76","Type":"ContainerStarted","Data":"7bfd063d0b7f949dcca3b65908238f0271e43462a25957c58a778d2cf534e6c3"}
Sep 11 12:56:28.714845 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:28.714794    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-ppzv7" podStartSLOduration=1.510396041 podStartE2EDuration="3.714775268s" totalImagesPullingTime="2.204379227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:28.713688666 +0000 UTC m=+42.653390000" watchObservedRunningTime="2026-09-11 12:56:28.714775268 +0000 UTC m=+42.654476604"
Sep 11 12:56:30.661658 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:30.661612    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:30.662100 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:30.661712    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:30.662100 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:30.661782    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 12:56:30.662100 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:30.661829    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 12:56:30.662100 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:30.661860    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls podName:0299ca51-7147-46ea-83f9-90e9efca03c7 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:38.661839045 +0000 UTC m=+52.601540361 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fvrvd" (UID: "0299ca51-7147-46ea-83f9-90e9efca03c7") : secret "insights-runtime-extractor-tls" not found
Sep 11 12:56:30.662100 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:30.661880    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm podName:0299ca51-7147-46ea-83f9-90e9efca03c7 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:38.661870192 +0000 UTC m=+52.601571506 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fvrvd" (UID: "0299ca51-7147-46ea-83f9-90e9efca03c7") : configmap "kube-rbac-proxy" not found
Sep 11 12:56:35.901169 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:35.901132    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:35.903584 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:35.903557    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60ebfb6b-c42a-4634-ad12-24056202391e-metrics-tls\") pod \"dns-default-nt9jt\" (UID: \"60ebfb6b-c42a-4634-ad12-24056202391e\") " pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:36.203408 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:36.203347    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:36.502669 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:36.502643    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nt9jt"]
Sep 11 12:56:36.505245 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:56:36.505219    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60ebfb6b_c42a_4634_ad12_24056202391e.slice/crio-af9f25dab5f637d5cdf481e04246056eb62b5418166878d2fc882741c36b07e5 WatchSource:0}: Error finding container af9f25dab5f637d5cdf481e04246056eb62b5418166878d2fc882741c36b07e5: Status 404 returned error can't find the container with id af9f25dab5f637d5cdf481e04246056eb62b5418166878d2fc882741c36b07e5
Sep 11 12:56:36.717263 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:36.717240    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nt9jt" event={"ID":"60ebfb6b-c42a-4634-ad12-24056202391e","Type":"ContainerStarted","Data":"af9f25dab5f637d5cdf481e04246056eb62b5418166878d2fc882741c36b07e5"}
Sep 11 12:56:38.722469 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:38.722409    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:38.722867 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:38.722518    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:38.722867 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:38.722625    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 12:56:38.722867 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:38.722729    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls podName:0299ca51-7147-46ea-83f9-90e9efca03c7 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:54.722694975 +0000 UTC m=+68.662396303 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fvrvd" (UID: "0299ca51-7147-46ea-83f9-90e9efca03c7") : secret "insights-runtime-extractor-tls" not found
Sep 11 12:56:38.723155 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:38.723131    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0299ca51-7147-46ea-83f9-90e9efca03c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:38.726008 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:38.725983    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nt9jt" event={"ID":"60ebfb6b-c42a-4634-ad12-24056202391e","Type":"ContainerStarted","Data":"d18f21ebadd4ce2aa992ece5e86154c62d907482075f305b011ac014b69337d6"}
Sep 11 12:56:38.726103 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:38.726065    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nt9jt" event={"ID":"60ebfb6b-c42a-4634-ad12-24056202391e","Type":"ContainerStarted","Data":"8d04c8f1345f85d3c124152f833213d167c8792e82781f4a3798a2805286fcf4"}
Sep 11 12:56:38.726103 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:38.726095    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:38.751568 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:38.751531    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nt9jt" podStartSLOduration=17.881377231 podStartE2EDuration="19.751519381s" totalImagesPullingTime="1.87014215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:38.750156901 +0000 UTC m=+52.689858235" watchObservedRunningTime="2026-09-11 12:56:38.751519381 +0000 UTC m=+52.691220712"
Sep 11 12:56:44.656684 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:44.656657    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p9msj"
Sep 11 12:56:47.523306 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.523273    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-9d9zz"]
Sep 11 12:56:47.526046 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.526025    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-9d9zz"
Sep 11 12:56:47.533780 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.533755    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 11 12:56:47.533888 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.533766    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mll82\""
Sep 11 12:56:47.533888 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.533826    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 11 12:56:47.558266 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.558242    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-9d9zz"]
Sep 11 12:56:47.577436 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.577403    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba03dab9-0722-48d6-9441-87eacb436c12-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-9d9zz\" (UID: \"ba03dab9-0722-48d6-9441-87eacb436c12\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-9d9zz"
Sep 11 12:56:47.577544 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.577472    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ba03dab9-0722-48d6-9441-87eacb436c12-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-9d9zz\" (UID: \"ba03dab9-0722-48d6-9441-87eacb436c12\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-9d9zz"
Sep 11 12:56:47.678393 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.678366    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ba03dab9-0722-48d6-9441-87eacb436c12-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-9d9zz\" (UID: \"ba03dab9-0722-48d6-9441-87eacb436c12\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-9d9zz"
Sep 11 12:56:47.678524 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.678477    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba03dab9-0722-48d6-9441-87eacb436c12-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-9d9zz\" (UID: \"ba03dab9-0722-48d6-9441-87eacb436c12\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-9d9zz"
Sep 11 12:56:47.678966 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.678947    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ba03dab9-0722-48d6-9441-87eacb436c12-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-9d9zz\" (UID: \"ba03dab9-0722-48d6-9441-87eacb436c12\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-9d9zz"
Sep 11 12:56:47.680815 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.680791    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ba03dab9-0722-48d6-9441-87eacb436c12-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-9d9zz\" (UID: \"ba03dab9-0722-48d6-9441-87eacb436c12\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-9d9zz"
Sep 11 12:56:47.684200 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.684181    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f7b4f7974-hwkmf"]
Sep 11 12:56:47.686313 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.686299    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.691049 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.691029    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 12:56:47.691156 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.691079    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 12:56:47.691216 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.691170    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 12:56:47.695239 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.695221    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g4h6z\""
Sep 11 12:56:47.711256 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.711237    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 12:56:47.720690 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.720670    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f7b4f7974-hwkmf"]
Sep 11 12:56:47.778803 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.778742    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21a85115-2aac-4af5-b310-f6334c7ecaa5-image-registry-private-configuration\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.778803 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.778774    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21a85115-2aac-4af5-b310-f6334c7ecaa5-registry-certificates\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.778930 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.778874    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21a85115-2aac-4af5-b310-f6334c7ecaa5-ca-trust-extracted\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.778930 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.778902    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21a85115-2aac-4af5-b310-f6334c7ecaa5-trusted-ca\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.778930 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.778917    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21a85115-2aac-4af5-b310-f6334c7ecaa5-installation-pull-secrets\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.779039 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.778955    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21a85115-2aac-4af5-b310-f6334c7ecaa5-registry-tls\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.779039 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.778972    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21a85115-2aac-4af5-b310-f6334c7ecaa5-bound-sa-token\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.779039 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.778998    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkpj2\" (UniqueName: \"kubernetes.io/projected/21a85115-2aac-4af5-b310-f6334c7ecaa5-kube-api-access-hkpj2\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.834309 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.834285    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-9d9zz"
Sep 11 12:56:47.880040 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.880004    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21a85115-2aac-4af5-b310-f6334c7ecaa5-registry-tls\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.880173 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.880054    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21a85115-2aac-4af5-b310-f6334c7ecaa5-bound-sa-token\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.880173 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.880086    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkpj2\" (UniqueName: \"kubernetes.io/projected/21a85115-2aac-4af5-b310-f6334c7ecaa5-kube-api-access-hkpj2\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.880173 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.880158    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21a85115-2aac-4af5-b310-f6334c7ecaa5-image-registry-private-configuration\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.880338 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.880186    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21a85115-2aac-4af5-b310-f6334c7ecaa5-registry-certificates\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.880338 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.880257    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21a85115-2aac-4af5-b310-f6334c7ecaa5-ca-trust-extracted\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.880338 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.880287    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21a85115-2aac-4af5-b310-f6334c7ecaa5-trusted-ca\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.880338 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.880314    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21a85115-2aac-4af5-b310-f6334c7ecaa5-installation-pull-secrets\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.881118 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.880990    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21a85115-2aac-4af5-b310-f6334c7ecaa5-ca-trust-extracted\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.881379 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.881354    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21a85115-2aac-4af5-b310-f6334c7ecaa5-registry-certificates\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.882604 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.882588    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21a85115-2aac-4af5-b310-f6334c7ecaa5-trusted-ca\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.884533 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.884508    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21a85115-2aac-4af5-b310-f6334c7ecaa5-installation-pull-secrets\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.884700 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.884682    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21a85115-2aac-4af5-b310-f6334c7ecaa5-registry-tls\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.885484 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.885447    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21a85115-2aac-4af5-b310-f6334c7ecaa5-image-registry-private-configuration\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.889522 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.889500    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21a85115-2aac-4af5-b310-f6334c7ecaa5-bound-sa-token\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.897924 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.897877    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkpj2\" (UniqueName: \"kubernetes.io/projected/21a85115-2aac-4af5-b310-f6334c7ecaa5-kube-api-access-hkpj2\") pod \"image-registry-5f7b4f7974-hwkmf\" (UID: \"21a85115-2aac-4af5-b310-f6334c7ecaa5\") " pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:47.994915 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:47.994896    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:48.136373 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:48.136346    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-9d9zz"]
Sep 11 12:56:48.136735 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:56:48.136713    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba03dab9_0722_48d6_9441_87eacb436c12.slice/crio-80212e7b22822866c2c02cb6472263e3bb7bbf17fc16386e1740d2ee80801de9 WatchSource:0}: Error finding container 80212e7b22822866c2c02cb6472263e3bb7bbf17fc16386e1740d2ee80801de9: Status 404 returned error can't find the container with id 80212e7b22822866c2c02cb6472263e3bb7bbf17fc16386e1740d2ee80801de9
Sep 11 12:56:48.294187 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:48.294134    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f7b4f7974-hwkmf"]
Sep 11 12:56:48.296555 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:56:48.296532    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21a85115_2aac_4af5_b310_f6334c7ecaa5.slice/crio-be888ca0b759eedf71f53eae151557a4bc587cd66da7072c9e4e04d6366af0ba WatchSource:0}: Error finding container be888ca0b759eedf71f53eae151557a4bc587cd66da7072c9e4e04d6366af0ba: Status 404 returned error can't find the container with id be888ca0b759eedf71f53eae151557a4bc587cd66da7072c9e4e04d6366af0ba
Sep 11 12:56:48.732562 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:48.732534    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nt9jt"
Sep 11 12:56:48.760063 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:48.760033    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf" event={"ID":"21a85115-2aac-4af5-b310-f6334c7ecaa5","Type":"ContainerStarted","Data":"e42f882a32455d3a4f5d179e80568b3e640d499737bd51ff0ceec257777465dc"}
Sep 11 12:56:48.760361 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:48.760088    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:56:48.760361 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:48.760109    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf" event={"ID":"21a85115-2aac-4af5-b310-f6334c7ecaa5","Type":"ContainerStarted","Data":"be888ca0b759eedf71f53eae151557a4bc587cd66da7072c9e4e04d6366af0ba"}
Sep 11 12:56:48.761150 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:48.761130    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-9d9zz" event={"ID":"ba03dab9-0722-48d6-9441-87eacb436c12","Type":"ContainerStarted","Data":"80212e7b22822866c2c02cb6472263e3bb7bbf17fc16386e1740d2ee80801de9"}
Sep 11 12:56:48.880621 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:48.880568    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf" podStartSLOduration=1.8805468250000001 podStartE2EDuration="1.880546825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:48.879087175 +0000 UTC m=+62.818788500" watchObservedRunningTime="2026-09-11 12:56:48.880546825 +0000 UTC m=+62.820248160"
Sep 11 12:56:49.361480 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:49.361449    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-42vpt"]
Sep 11 12:56:49.363252 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:49.363225    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-42vpt"
Sep 11 12:56:49.369024 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:49.369000    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bqs2x\""
Sep 11 12:56:49.369415 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:49.369398    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 11 12:56:49.387629 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:49.387608    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-42vpt"]
Sep 11 12:56:49.491032 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:49.491009    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bc85ed6c-96aa-4ca5-839e-073142c4b699-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-42vpt\" (UID: \"bc85ed6c-96aa-4ca5-839e-073142c4b699\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-42vpt"
Sep 11 12:56:49.591558 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:49.591531    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bc85ed6c-96aa-4ca5-839e-073142c4b699-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-42vpt\" (UID: \"bc85ed6c-96aa-4ca5-839e-073142c4b699\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-42vpt"
Sep 11 12:56:49.593679 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:49.593653    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bc85ed6c-96aa-4ca5-839e-073142c4b699-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-42vpt\" (UID: \"bc85ed6c-96aa-4ca5-839e-073142c4b699\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-42vpt"
Sep 11 12:56:49.672940 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:49.672896    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-42vpt"
Sep 11 12:56:49.765768 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:49.765733    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-9d9zz" event={"ID":"ba03dab9-0722-48d6-9441-87eacb436c12","Type":"ContainerStarted","Data":"b7e6a503853c5bb3ab21982e647ae56c6b7beb96bc886dcc6ee8efd5bee184af"}
Sep 11 12:56:49.789023 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:49.788982    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-9d9zz" podStartSLOduration=1.5552956180000002 podStartE2EDuration="2.788966627s" totalImagesPullingTime="1.233671009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:49.787729621 +0000 UTC m=+63.727430955" watchObservedRunningTime="2026-09-11 12:56:49.788966627 +0000 UTC m=+63.728667958"
Sep 11 12:56:49.953326 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:49.953258    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-42vpt"]
Sep 11 12:56:49.957400 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:56:49.957377    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc85ed6c_96aa_4ca5_839e_073142c4b699.slice/crio-66ae8e13afb67f05d5717bfdbce6ef4a4e1d0478a0aa6a9d3f496dc388846876 WatchSource:0}: Error finding container 66ae8e13afb67f05d5717bfdbce6ef4a4e1d0478a0aa6a9d3f496dc388846876: Status 404 returned error can't find the container with id 66ae8e13afb67f05d5717bfdbce6ef4a4e1d0478a0aa6a9d3f496dc388846876
Sep 11 12:56:50.770439 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:50.770392    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-42vpt" event={"ID":"bc85ed6c-96aa-4ca5-839e-073142c4b699","Type":"ContainerStarted","Data":"66ae8e13afb67f05d5717bfdbce6ef4a4e1d0478a0aa6a9d3f496dc388846876"}
Sep 11 12:56:51.775162 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:51.775098    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-42vpt" event={"ID":"bc85ed6c-96aa-4ca5-839e-073142c4b699","Type":"ContainerStarted","Data":"1eeff90ef8d45bb460d6ee33932d728f32ff872f87f2218bec1fc4f670f58d3e"}
Sep 11 12:56:51.775476 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:51.775310    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-42vpt"
Sep 11 12:56:51.779896 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:51.779873    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-42vpt"
Sep 11 12:56:51.806565 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:51.806524    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-42vpt" podStartSLOduration=1.287440392 podStartE2EDuration="2.80650963s" totalImagesPullingTime="1.519069238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:51.805917596 +0000 UTC m=+65.745618929" watchObservedRunningTime="2026-09-11 12:56:51.80650963 +0000 UTC m=+65.746210963"
Sep 11 12:56:52.312241 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.312209    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:52.312241 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.312243    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs\") pod \"network-metrics-daemon-g5nkq\" (UID: \"4c721b01-f754-4b83-87fe-94689fb629c9\") " pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:52.316273 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.316248    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 12:56:52.317001 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.316982    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 12:56:52.325841 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.325811    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c721b01-f754-4b83-87fe-94689fb629c9-metrics-certs\") pod \"network-metrics-daemon-g5nkq\" (UID: \"4c721b01-f754-4b83-87fe-94689fb629c9\") " pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:52.325933 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.325861    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/da6d1a19-06a1-4a7a-bbc2-593b438eeb41-original-pull-secret\") pod \"global-pull-secret-syncer-lbn4f\" (UID: \"da6d1a19-06a1-4a7a-bbc2-593b438eeb41\") " pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:52.378426 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.378402    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbn4f"
Sep 11 12:56:52.413312 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.413285    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9r54\" (UniqueName: \"kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54\") pod \"network-check-target-6k8nb\" (UID: \"0a1899d1-b90c-4b2f-b49c-06c3a2e3533d\") " pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:52.416317 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.416294    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 12:56:52.428660 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.428643    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 12:56:52.437196 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.437176    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9r54\" (UniqueName: \"kubernetes.io/projected/0a1899d1-b90c-4b2f-b49c-06c3a2e3533d-kube-api-access-t9r54\") pod \"network-check-target-6k8nb\" (UID: \"0a1899d1-b90c-4b2f-b49c-06c3a2e3533d\") " pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:52.583202 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.583147    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w7fbx\""
Sep 11 12:56:52.590860 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.590839    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g5nkq"
Sep 11 12:56:52.593225 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.593203    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"]
Sep 11 12:56:52.596291 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.596275    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"
Sep 11 12:56:52.602863 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.602783    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 11 12:56:52.603058 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.603042    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-m4tc2\""
Sep 11 12:56:52.603175 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.603159    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 12:56:52.603632 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.603605    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 12:56:52.604052 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.604032    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 12:56:52.604148 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.604056    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 11 12:56:52.637221 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.637196    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7r4pp\""
Sep 11 12:56:52.640717 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.640520    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:52.642169 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.642146    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"]
Sep 11 12:56:52.676444 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.676400    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lbn4f"]
Sep 11 12:56:52.683509 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:56:52.683481    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda6d1a19_06a1_4a7a_bbc2_593b438eeb41.slice/crio-3e4056df5ea04db2182c5659b46b32b3f5f58a7c2ef00c1f2b30dacaea2a5485 WatchSource:0}: Error finding container 3e4056df5ea04db2182c5659b46b32b3f5f58a7c2ef00c1f2b30dacaea2a5485: Status 404 returned error can't find the container with id 3e4056df5ea04db2182c5659b46b32b3f5f58a7c2ef00c1f2b30dacaea2a5485
Sep 11 12:56:52.715902 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.715879    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-hdbdn\" (UID: \"78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"
Sep 11 12:56:52.716006 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.715922    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rb7q\" (UniqueName: \"kubernetes.io/projected/78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d-kube-api-access-6rb7q\") pod \"prometheus-operator-68bcd7f987-hdbdn\" (UID: \"78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"
Sep 11 12:56:52.716054 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.716012    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-hdbdn\" (UID: \"78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"
Sep 11 12:56:52.716054 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.716039    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-hdbdn\" (UID: \"78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"
Sep 11 12:56:52.779570 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.779536    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lbn4f" event={"ID":"da6d1a19-06a1-4a7a-bbc2-593b438eeb41","Type":"ContainerStarted","Data":"3e4056df5ea04db2182c5659b46b32b3f5f58a7c2ef00c1f2b30dacaea2a5485"}
Sep 11 12:56:52.816551 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.816530    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rb7q\" (UniqueName: \"kubernetes.io/projected/78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d-kube-api-access-6rb7q\") pod \"prometheus-operator-68bcd7f987-hdbdn\" (UID: \"78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"
Sep 11 12:56:52.816626 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.816576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-hdbdn\" (UID: \"78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"
Sep 11 12:56:52.816626 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.816601    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-hdbdn\" (UID: \"78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"
Sep 11 12:56:52.816705 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.816663    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-hdbdn\" (UID: \"78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"
Sep 11 12:56:52.817213 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.817193    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-hdbdn\" (UID: \"78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"
Sep 11 12:56:52.818726 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.818701    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-hdbdn\" (UID: \"78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"
Sep 11 12:56:52.818797 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.818777    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-hdbdn\" (UID: \"78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"
Sep 11 12:56:52.837629 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.837588    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rb7q\" (UniqueName: \"kubernetes.io/projected/78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d-kube-api-access-6rb7q\") pod \"prometheus-operator-68bcd7f987-hdbdn\" (UID: \"78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"
Sep 11 12:56:52.874324 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.874290    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g5nkq"]
Sep 11 12:56:52.881095 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:56:52.881075    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c721b01_f754_4b83_87fe_94689fb629c9.slice/crio-4343890ca32e33070f61dea83420174124af2ace37aa1ff9f9a3edda103984c7 WatchSource:0}: Error finding container 4343890ca32e33070f61dea83420174124af2ace37aa1ff9f9a3edda103984c7: Status 404 returned error can't find the container with id 4343890ca32e33070f61dea83420174124af2ace37aa1ff9f9a3edda103984c7
Sep 11 12:56:52.915564 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.915545    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"
Sep 11 12:56:52.933277 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:52.933258    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6k8nb"]
Sep 11 12:56:52.936136 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:56:52.936117    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a1899d1_b90c_4b2f_b49c_06c3a2e3533d.slice/crio-7044cb18930087feb5ae017552acd816b3424b91d53c835405224c60f602458e WatchSource:0}: Error finding container 7044cb18930087feb5ae017552acd816b3424b91d53c835405224c60f602458e: Status 404 returned error can't find the container with id 7044cb18930087feb5ae017552acd816b3424b91d53c835405224c60f602458e
Sep 11 12:56:53.200155 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:53.200083    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn"]
Sep 11 12:56:53.203970 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:56:53.203940    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78e0c53c_3e9a_46aa_9f67_ee1c8fa6577d.slice/crio-317e3d27081add45c6b8c9bbe9ae0c86fad19d5c9f8e1b5b83dec44d1603f27d WatchSource:0}: Error finding container 317e3d27081add45c6b8c9bbe9ae0c86fad19d5c9f8e1b5b83dec44d1603f27d: Status 404 returned error can't find the container with id 317e3d27081add45c6b8c9bbe9ae0c86fad19d5c9f8e1b5b83dec44d1603f27d
Sep 11 12:56:53.786261 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:53.786218    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn" event={"ID":"78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d","Type":"ContainerStarted","Data":"317e3d27081add45c6b8c9bbe9ae0c86fad19d5c9f8e1b5b83dec44d1603f27d"}
Sep 11 12:56:53.787651 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:53.787625    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6k8nb" event={"ID":"0a1899d1-b90c-4b2f-b49c-06c3a2e3533d","Type":"ContainerStarted","Data":"7044cb18930087feb5ae017552acd816b3424b91d53c835405224c60f602458e"}
Sep 11 12:56:53.789618 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:53.789563    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g5nkq" event={"ID":"4c721b01-f754-4b83-87fe-94689fb629c9","Type":"ContainerStarted","Data":"4343890ca32e33070f61dea83420174124af2ace37aa1ff9f9a3edda103984c7"}
Sep 11 12:56:54.733523 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:54.733495    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:54.736331 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:54.736297    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0299ca51-7147-46ea-83f9-90e9efca03c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvrvd\" (UID: \"0299ca51-7147-46ea-83f9-90e9efca03c7\") " pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:54.796466 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:54.796433    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g5nkq" event={"ID":"4c721b01-f754-4b83-87fe-94689fb629c9","Type":"ContainerStarted","Data":"a38fd1814d815feb3e385849d6d5977a6bff504b264e7f65f5dab286f6411a17"}
Sep 11 12:56:55.038037 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:55.037772    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-22vc9\""
Sep 11 12:56:55.045460 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:55.044972    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fvrvd"
Sep 11 12:56:55.353042 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:55.352973    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fvrvd"]
Sep 11 12:56:55.356191 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:56:55.356164    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0299ca51_7147_46ea_83f9_90e9efca03c7.slice/crio-2fe3c20e64e5c765a65028df1ee36a6427f8a70076942eb3a5ba28ccbe4066b0 WatchSource:0}: Error finding container 2fe3c20e64e5c765a65028df1ee36a6427f8a70076942eb3a5ba28ccbe4066b0: Status 404 returned error can't find the container with id 2fe3c20e64e5c765a65028df1ee36a6427f8a70076942eb3a5ba28ccbe4066b0
Sep 11 12:56:55.802581 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:55.802323    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn" event={"ID":"78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d","Type":"ContainerStarted","Data":"907dd668e300aa7f40f4f579f3999b4e7741e46a13dcc5371863c1a7f69c999f"}
Sep 11 12:56:55.802581 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:55.802390    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn" event={"ID":"78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d","Type":"ContainerStarted","Data":"afe0355e9b8f3766a73961311898a5ddb3adda0f6fd622cc2a8f55116de1080e"}
Sep 11 12:56:55.807843 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:55.807805    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fvrvd" event={"ID":"0299ca51-7147-46ea-83f9-90e9efca03c7","Type":"ContainerStarted","Data":"b58a5e6b74936c07f23af5c307ca4f717bcc5a24e1faa269d4df0ab2871b9b05"}
Sep 11 12:56:55.807974 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:55.807890    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fvrvd" event={"ID":"0299ca51-7147-46ea-83f9-90e9efca03c7","Type":"ContainerStarted","Data":"2fe3c20e64e5c765a65028df1ee36a6427f8a70076942eb3a5ba28ccbe4066b0"}
Sep 11 12:56:55.810030 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:55.810002    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g5nkq" event={"ID":"4c721b01-f754-4b83-87fe-94689fb629c9","Type":"ContainerStarted","Data":"18b2bc3eddba7f677b19701ba2d39434b4ac8a15ae4e51d0742c379e2e147c7b"}
Sep 11 12:56:55.883965 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:55.883923    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hdbdn" podStartSLOduration=2.292957376 podStartE2EDuration="3.883909434s" totalImagesPullingTime="1.590952058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:55.831597489 +0000 UTC m=+69.771298832" watchObservedRunningTime="2026-09-11 12:56:55.883909434 +0000 UTC m=+69.823610763"
Sep 11 12:56:55.884107 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:55.884092    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-g5nkq" podStartSLOduration=68.507453447 podStartE2EDuration="1m9.884087044s" totalImagesPullingTime="1.376633597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:55.882510083 +0000 UTC m=+69.822211416" watchObservedRunningTime="2026-09-11 12:56:55.884087044 +0000 UTC m=+69.823788375"
Sep 11 12:56:58.274750 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.274726    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"]
Sep 11 12:56:58.291709 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.291690    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"
Sep 11 12:56:58.295698 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.295648    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-rwrxq\""
Sep 11 12:56:58.297566 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.297543    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 12:56:58.297659 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.297634    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 12:56:58.326822 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.326799    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"]
Sep 11 12:56:58.350641 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.350619    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ls8jx"]
Sep 11 12:56:58.356814 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.356794    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/439f4bb3-3bc7-40f9-8c92-9110801fc1f1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-7xl7v\" (UID: \"439f4bb3-3bc7-40f9-8c92-9110801fc1f1\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"
Sep 11 12:56:58.356890 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.356826    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/439f4bb3-3bc7-40f9-8c92-9110801fc1f1-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-7xl7v\" (UID: \"439f4bb3-3bc7-40f9-8c92-9110801fc1f1\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"
Sep 11 12:56:58.356890 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.356848    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm2r8\" (UniqueName: \"kubernetes.io/projected/439f4bb3-3bc7-40f9-8c92-9110801fc1f1-kube-api-access-nm2r8\") pod \"openshift-state-metrics-67dcf8c979-7xl7v\" (UID: \"439f4bb3-3bc7-40f9-8c92-9110801fc1f1\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"
Sep 11 12:56:58.356999 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.356935    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/439f4bb3-3bc7-40f9-8c92-9110801fc1f1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-7xl7v\" (UID: \"439f4bb3-3bc7-40f9-8c92-9110801fc1f1\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"
Sep 11 12:56:58.359676 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.359658    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.362678 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.362660    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 12:56:58.368843 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.368823    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4jg94\""
Sep 11 12:56:58.369157 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.369141    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 12:56:58.369258 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.369232    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 12:56:58.458298 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.458275    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/439f4bb3-3bc7-40f9-8c92-9110801fc1f1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-7xl7v\" (UID: \"439f4bb3-3bc7-40f9-8c92-9110801fc1f1\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"
Sep 11 12:56:58.458410 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.458315    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fe281be4-697f-4ff8-8ab7-153c0987e10d-node-exporter-tls\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.458410 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.458348    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe281be4-697f-4ff8-8ab7-153c0987e10d-metrics-client-ca\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.458410 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.458374    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe281be4-697f-4ff8-8ab7-153c0987e10d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.458410 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.458403    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fe281be4-697f-4ff8-8ab7-153c0987e10d-root\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.458637 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:58.458413    2582 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 11 12:56:58.458637 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.458448    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fe281be4-697f-4ff8-8ab7-153c0987e10d-node-exporter-wtmp\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.458637 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:56:58.458497    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/439f4bb3-3bc7-40f9-8c92-9110801fc1f1-openshift-state-metrics-tls podName:439f4bb3-3bc7-40f9-8c92-9110801fc1f1 nodeName:}" failed. No retries permitted until 2026-09-11 12:56:58.958477054 +0000 UTC m=+72.898178374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/439f4bb3-3bc7-40f9-8c92-9110801fc1f1-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-7xl7v" (UID: "439f4bb3-3bc7-40f9-8c92-9110801fc1f1") : secret "openshift-state-metrics-tls" not found
Sep 11 12:56:58.458637 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.458520    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2x8d\" (UniqueName: \"kubernetes.io/projected/fe281be4-697f-4ff8-8ab7-153c0987e10d-kube-api-access-w2x8d\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.458637 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.458557    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/439f4bb3-3bc7-40f9-8c92-9110801fc1f1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-7xl7v\" (UID: \"439f4bb3-3bc7-40f9-8c92-9110801fc1f1\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"
Sep 11 12:56:58.458637 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.458585    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fe281be4-697f-4ff8-8ab7-153c0987e10d-node-exporter-accelerators-collector-config\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.458637 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.458617    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/439f4bb3-3bc7-40f9-8c92-9110801fc1f1-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-7xl7v\" (UID: \"439f4bb3-3bc7-40f9-8c92-9110801fc1f1\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"
Sep 11 12:56:58.459008 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.458649    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm2r8\" (UniqueName: \"kubernetes.io/projected/439f4bb3-3bc7-40f9-8c92-9110801fc1f1-kube-api-access-nm2r8\") pod \"openshift-state-metrics-67dcf8c979-7xl7v\" (UID: \"439f4bb3-3bc7-40f9-8c92-9110801fc1f1\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"
Sep 11 12:56:58.459008 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.458675    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fe281be4-697f-4ff8-8ab7-153c0987e10d-node-exporter-textfile\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.459008 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.458694    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fe281be4-697f-4ff8-8ab7-153c0987e10d-sys\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.459374 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.459354    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/439f4bb3-3bc7-40f9-8c92-9110801fc1f1-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-7xl7v\" (UID: \"439f4bb3-3bc7-40f9-8c92-9110801fc1f1\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"
Sep 11 12:56:58.461232 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.461210    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/439f4bb3-3bc7-40f9-8c92-9110801fc1f1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-7xl7v\" (UID: \"439f4bb3-3bc7-40f9-8c92-9110801fc1f1\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"
Sep 11 12:56:58.483773 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.483745    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm2r8\" (UniqueName: \"kubernetes.io/projected/439f4bb3-3bc7-40f9-8c92-9110801fc1f1-kube-api-access-nm2r8\") pod \"openshift-state-metrics-67dcf8c979-7xl7v\" (UID: \"439f4bb3-3bc7-40f9-8c92-9110801fc1f1\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"
Sep 11 12:56:58.559656 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.559626    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fe281be4-697f-4ff8-8ab7-153c0987e10d-sys\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.559788 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.559700    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fe281be4-697f-4ff8-8ab7-153c0987e10d-node-exporter-tls\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.559788 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.559732    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe281be4-697f-4ff8-8ab7-153c0987e10d-metrics-client-ca\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.559788 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.559745    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fe281be4-697f-4ff8-8ab7-153c0987e10d-sys\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.559788 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.559756    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe281be4-697f-4ff8-8ab7-153c0987e10d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.559970 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.559828    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fe281be4-697f-4ff8-8ab7-153c0987e10d-root\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.559970 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.559870    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fe281be4-697f-4ff8-8ab7-153c0987e10d-node-exporter-wtmp\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.559970 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.559892    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2x8d\" (UniqueName: \"kubernetes.io/projected/fe281be4-697f-4ff8-8ab7-153c0987e10d-kube-api-access-w2x8d\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.559970 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.559938    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fe281be4-697f-4ff8-8ab7-153c0987e10d-root\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.560148 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.559943    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fe281be4-697f-4ff8-8ab7-153c0987e10d-node-exporter-accelerators-collector-config\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.560148 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.560038    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fe281be4-697f-4ff8-8ab7-153c0987e10d-node-exporter-wtmp\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.560148 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.560078    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fe281be4-697f-4ff8-8ab7-153c0987e10d-node-exporter-textfile\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.560460 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.560411    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fe281be4-697f-4ff8-8ab7-153c0987e10d-node-exporter-textfile\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.560460 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.560441    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe281be4-697f-4ff8-8ab7-153c0987e10d-metrics-client-ca\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.560646 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.560623    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fe281be4-697f-4ff8-8ab7-153c0987e10d-node-exporter-accelerators-collector-config\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.562356 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.562333    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe281be4-697f-4ff8-8ab7-153c0987e10d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.562475 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.562447    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fe281be4-697f-4ff8-8ab7-153c0987e10d-node-exporter-tls\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.571292 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.571270    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2x8d\" (UniqueName: \"kubernetes.io/projected/fe281be4-697f-4ff8-8ab7-153c0987e10d-kube-api-access-w2x8d\") pod \"node-exporter-ls8jx\" (UID: \"fe281be4-697f-4ff8-8ab7-153c0987e10d\") " pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.670508 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.670476    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ls8jx"
Sep 11 12:56:58.764318 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:56:58.764288    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe281be4_697f_4ff8_8ab7_153c0987e10d.slice/crio-7d677838f530eebbf81ba7a60a8d1bafb65bdd3f56b9092d7782c0521812d9c7 WatchSource:0}: Error finding container 7d677838f530eebbf81ba7a60a8d1bafb65bdd3f56b9092d7782c0521812d9c7: Status 404 returned error can't find the container with id 7d677838f530eebbf81ba7a60a8d1bafb65bdd3f56b9092d7782c0521812d9c7
Sep 11 12:56:58.769950 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.769929    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-2zvkx"]
Sep 11 12:56:58.797489 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.797464    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-2zvkx"
Sep 11 12:56:58.808911 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.808883    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 12:56:58.809136 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.809114    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-2zvkx"]
Sep 11 12:56:58.811807 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.811791    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ctj89\""
Sep 11 12:56:58.812835 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.812818    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 12:56:58.821968 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.821941    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ls8jx" event={"ID":"fe281be4-697f-4ff8-8ab7-153c0987e10d","Type":"ContainerStarted","Data":"7d677838f530eebbf81ba7a60a8d1bafb65bdd3f56b9092d7782c0521812d9c7"}
Sep 11 12:56:58.823643 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.823589    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lbn4f" event={"ID":"da6d1a19-06a1-4a7a-bbc2-593b438eeb41","Type":"ContainerStarted","Data":"fa2dbf765cc61a1f4c66dafe7e5f1a75299ac31a6d7b81092a8d242be8a76454"}
Sep 11 12:56:58.825187 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.825159    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6k8nb" event={"ID":"0a1899d1-b90c-4b2f-b49c-06c3a2e3533d","Type":"ContainerStarted","Data":"6f9ffe2e1a445a0f8110ea55eee799c29e53cd89a20ae4d0d8a99c2b6a93da3e"}
Sep 11 12:56:58.825304 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.825289    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:56:58.862433 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.862396    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd2xp\" (UniqueName: \"kubernetes.io/projected/7a54332c-5c8c-4224-ad6b-e9e4be156923-kube-api-access-jd2xp\") pod \"downloads-76f99584f4-2zvkx\" (UID: \"7a54332c-5c8c-4224-ad6b-e9e4be156923\") " pod="openshift-console/downloads-76f99584f4-2zvkx"
Sep 11 12:56:58.880244 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.880195    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6k8nb" podStartSLOduration=67.428735321 podStartE2EDuration="1m12.880177357s" totalImagesPullingTime="5.451442036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:58.880045684 +0000 UTC m=+72.819747017" watchObservedRunningTime="2026-09-11 12:56:58.880177357 +0000 UTC m=+72.819878692"
Sep 11 12:56:58.916727 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.916678    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lbn4f" podStartSLOduration=67.213326834 podStartE2EDuration="1m12.916659954s" totalImagesPullingTime="5.70333312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:56:58.915058229 +0000 UTC m=+72.854759562" watchObservedRunningTime="2026-09-11 12:56:58.916659954 +0000 UTC m=+72.856361287"
Sep 11 12:56:58.963642 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.963619    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd2xp\" (UniqueName: \"kubernetes.io/projected/7a54332c-5c8c-4224-ad6b-e9e4be156923-kube-api-access-jd2xp\") pod \"downloads-76f99584f4-2zvkx\" (UID: \"7a54332c-5c8c-4224-ad6b-e9e4be156923\") " pod="openshift-console/downloads-76f99584f4-2zvkx"
Sep 11 12:56:58.963747 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.963663    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/439f4bb3-3bc7-40f9-8c92-9110801fc1f1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-7xl7v\" (UID: \"439f4bb3-3bc7-40f9-8c92-9110801fc1f1\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"
Sep 11 12:56:58.965846 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.965823    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/439f4bb3-3bc7-40f9-8c92-9110801fc1f1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-7xl7v\" (UID: \"439f4bb3-3bc7-40f9-8c92-9110801fc1f1\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"
Sep 11 12:56:58.995786 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:58.995766    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd2xp\" (UniqueName: \"kubernetes.io/projected/7a54332c-5c8c-4224-ad6b-e9e4be156923-kube-api-access-jd2xp\") pod \"downloads-76f99584f4-2zvkx\" (UID: \"7a54332c-5c8c-4224-ad6b-e9e4be156923\") " pod="openshift-console/downloads-76f99584f4-2zvkx"
Sep 11 12:56:59.106935 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.106878    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-2zvkx"
Sep 11 12:56:59.202236 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.202202    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"
Sep 11 12:56:59.411901 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.411757    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-2zvkx"]
Sep 11 12:56:59.415163 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:56:59.415133    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a54332c_5c8c_4224_ad6b_e9e4be156923.slice/crio-f96b7334062c003776193446aba52c6fc52441f9f50427a3f008d43d63888d58 WatchSource:0}: Error finding container f96b7334062c003776193446aba52c6fc52441f9f50427a3f008d43d63888d58: Status 404 returned error can't find the container with id f96b7334062c003776193446aba52c6fc52441f9f50427a3f008d43d63888d58
Sep 11 12:56:59.515867 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.515838    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v"]
Sep 11 12:56:59.722399 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.722356    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 12:56:59.726275 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.726244    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.734434 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.734395    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 12:56:59.734768 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.734733    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 12:56:59.735272 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.735044    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rzdbn\""
Sep 11 12:56:59.735531 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.735489    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 12:56:59.736064 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.735874    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 12:56:59.736064 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.735877    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 12:56:59.739214 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:56:59.739190    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod439f4bb3_3bc7_40f9_8c92_9110801fc1f1.slice/crio-8bcd30c9cf3eb89c5a4bf644f7f6ad1b82b91e6acf5bb1671e52fc98c56216a2 WatchSource:0}: Error finding container 8bcd30c9cf3eb89c5a4bf644f7f6ad1b82b91e6acf5bb1671e52fc98c56216a2: Status 404 returned error can't find the container with id 8bcd30c9cf3eb89c5a4bf644f7f6ad1b82b91e6acf5bb1671e52fc98c56216a2
Sep 11 12:56:59.740195 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.740093    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 12:56:59.740195 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.740130    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 12:56:59.744246 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.744224    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 12:56:59.751926 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.751903    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 12:56:59.770172 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.770148    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 12:56:59.772683 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.772660    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-config-out\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.772774 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.772698    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.772774 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.772734    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.772900 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.772803    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.772900 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.772830    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.772900 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.772853    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-web-config\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.773029 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.772906    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.773029 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.772962    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-config-volume\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.773029 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.772984    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdgd7\" (UniqueName: \"kubernetes.io/projected/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-kube-api-access-jdgd7\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.773166 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.773028    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.773166 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.773070    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.773166 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.773100    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.773166 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.773126    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.831619 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.831585    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fvrvd" event={"ID":"0299ca51-7147-46ea-83f9-90e9efca03c7","Type":"ContainerStarted","Data":"a995fb2104d02770fa45bdf2d41c7f82fa16e0d39db356ea2b0dc690c12b2171"}
Sep 11 12:56:59.833950 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.833521    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v" event={"ID":"439f4bb3-3bc7-40f9-8c92-9110801fc1f1","Type":"ContainerStarted","Data":"8bcd30c9cf3eb89c5a4bf644f7f6ad1b82b91e6acf5bb1671e52fc98c56216a2"}
Sep 11 12:56:59.840480 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.840450    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-2zvkx" event={"ID":"7a54332c-5c8c-4224-ad6b-e9e4be156923","Type":"ContainerStarted","Data":"f96b7334062c003776193446aba52c6fc52441f9f50427a3f008d43d63888d58"}
Sep 11 12:56:59.874426 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.874396    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.874526 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.874483    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-config-volume\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.874526 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.874509    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdgd7\" (UniqueName: \"kubernetes.io/projected/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-kube-api-access-jdgd7\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.874629 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.874547    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.874629 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.874578    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.875136 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.874742    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.875136 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.874792    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.875136 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.874820    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-config-out\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.875333 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.875184    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.875333 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.875226    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.875333 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.875309    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.875495 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.875350    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.875495 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.875381    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.875495 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.875407    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.875495 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.875448    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-web-config\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.877919 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.877441    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.877919 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.877673    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.877919 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.877880    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-config-volume\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.879132 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.878379    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.879587 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.879524    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.879698 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.879676    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-config-out\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.879762 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.879707    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.879901 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.879872    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.880381 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.880358    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.880498 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.880477    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-web-config\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:56:59.900254 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:56:59.900230    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdgd7\" (UniqueName: \"kubernetes.io/projected/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-kube-api-access-jdgd7\") pod \"alertmanager-main-0\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:57:00.037868 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:00.037781    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:57:00.507566 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:00.507462    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 12:57:00.510100 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:57:00.510057    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09ed0d59_d217_405b_9c84_e7b7ac2dc1bf.slice/crio-b48300d5983707afe8cab24e131366c5f2c5bdec34a49f1376024845134c5b3c WatchSource:0}: Error finding container b48300d5983707afe8cab24e131366c5f2c5bdec34a49f1376024845134c5b3c: Status 404 returned error can't find the container with id b48300d5983707afe8cab24e131366c5f2c5bdec34a49f1376024845134c5b3c
Sep 11 12:57:00.848390 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:00.848347    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fvrvd" event={"ID":"0299ca51-7147-46ea-83f9-90e9efca03c7","Type":"ContainerStarted","Data":"49708af64756eb841dc90114a38b5126889e5db19305fb77448566d3f942438b"}
Sep 11 12:57:00.850490 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:00.850451    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v" event={"ID":"439f4bb3-3bc7-40f9-8c92-9110801fc1f1","Type":"ContainerStarted","Data":"b801fc8634e9120d96b2367de21428dc87fd5857202976597b72261bc97b3324"}
Sep 11 12:57:00.850627 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:00.850509    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v" event={"ID":"439f4bb3-3bc7-40f9-8c92-9110801fc1f1","Type":"ContainerStarted","Data":"5ea68cc5aeb3ab3a316f5552d42f1143039c15986a787602743267159b87746b"}
Sep 11 12:57:00.851865 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:00.851838    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf","Type":"ContainerStarted","Data":"b48300d5983707afe8cab24e131366c5f2c5bdec34a49f1376024845134c5b3c"}
Sep 11 12:57:00.853654 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:00.853627    2582 generic.go:356] "Generic (PLEG): container finished" podID="fe281be4-697f-4ff8-8ab7-153c0987e10d" containerID="2d18463c1f56faabf3d68a6d2237a1101d94129c94c692f6eec21b3259a25be7" exitCode=0
Sep 11 12:57:00.853795 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:00.853683    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ls8jx" event={"ID":"fe281be4-697f-4ff8-8ab7-153c0987e10d","Type":"ContainerDied","Data":"2d18463c1f56faabf3d68a6d2237a1101d94129c94c692f6eec21b3259a25be7"}
Sep 11 12:57:00.885172 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:00.885118    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fvrvd" podStartSLOduration=34.226379135 podStartE2EDuration="38.885098602s" totalImagesPullingTime="4.658719467s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:56:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:57:00.883675858 +0000 UTC m=+74.823377192" watchObservedRunningTime="2026-09-11 12:57:00.885098602 +0000 UTC m=+74.824799936"
Sep 11 12:57:01.861667 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:01.861632    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ls8jx" event={"ID":"fe281be4-697f-4ff8-8ab7-153c0987e10d","Type":"ContainerStarted","Data":"51af87692b38857aae2ebf8a1b9949e0145ace115c70b0ac84e6f18c8653cd8d"}
Sep 11 12:57:01.862032 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:01.861700    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ls8jx" event={"ID":"fe281be4-697f-4ff8-8ab7-153c0987e10d","Type":"ContainerStarted","Data":"1cd301f0fd9ea05153431ba1574c1e68a1b8ec827080905dc7867e0307ea8e7b"}
Sep 11 12:57:01.931661 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:01.931610    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ls8jx" podStartSLOduration=2.468737443 podStartE2EDuration="3.93158952s" totalImagesPullingTime="1.395711877s" totalInitContainerRuntime="67.1402ms" isStatefulPod=true podCreationTimestamp="2026-09-11 12:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:57:01.930474124 +0000 UTC m=+75.870175457" watchObservedRunningTime="2026-09-11 12:57:01.93158952 +0000 UTC m=+75.871290853"
Sep 11 12:57:02.542336 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.542302    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-bb957f59b-2nsgs"]
Sep 11 12:57:02.548223 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.548191    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.551819 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.551787    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 11 12:57:02.552207 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.552185    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 11 12:57:02.552207 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.552201    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7p94z\""
Sep 11 12:57:02.552332 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.552243    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 11 12:57:02.552591 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.552569    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 11 12:57:02.552591 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.552587    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 11 12:57:02.552873 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.552849    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6sg5o1omvgp9l\""
Sep 11 12:57:02.599782 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.599761    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p2w9\" (UniqueName: \"kubernetes.io/projected/126a62b4-8979-4c34-a839-bb18421a90a7-kube-api-access-2p2w9\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.599893 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.599800    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-grpc-tls\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.599893 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.599865    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/126a62b4-8979-4c34-a839-bb18421a90a7-metrics-client-ca\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.600006 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.599914    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.600006 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.599955    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.600006 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.599985    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-thanos-querier-tls\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.600144 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.600010    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.600144 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.600039    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.627759 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.627736    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-bb957f59b-2nsgs"]
Sep 11 12:57:02.700737 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.700692    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.700882 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.700743    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.700882 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.700773    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-thanos-querier-tls\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.700882 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.700794    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.700882 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.700814    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.701094 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.700889    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p2w9\" (UniqueName: \"kubernetes.io/projected/126a62b4-8979-4c34-a839-bb18421a90a7-kube-api-access-2p2w9\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.701094 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.700926    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-grpc-tls\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.701094 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.701002    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/126a62b4-8979-4c34-a839-bb18421a90a7-metrics-client-ca\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.701881 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.701819    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/126a62b4-8979-4c34-a839-bb18421a90a7-metrics-client-ca\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.703547 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.703519    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.703663 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.703631    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.703920 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.703893    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.704008 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.703893    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.704741 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.704717    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-grpc-tls\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.704872 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.704855    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/126a62b4-8979-4c34-a839-bb18421a90a7-secret-thanos-querier-tls\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.716020 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.715992    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p2w9\" (UniqueName: \"kubernetes.io/projected/126a62b4-8979-4c34-a839-bb18421a90a7-kube-api-access-2p2w9\") pod \"thanos-querier-bb957f59b-2nsgs\" (UID: \"126a62b4-8979-4c34-a839-bb18421a90a7\") " pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.859145 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.859054    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:02.869166 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.868863    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v" event={"ID":"439f4bb3-3bc7-40f9-8c92-9110801fc1f1","Type":"ContainerStarted","Data":"7fce459f0d152cb2e77fce52dda22e097a4c3ac9d2bdb8ad8e1da4c5ddf3a53a"}
Sep 11 12:57:02.871108 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.871081    2582 generic.go:356] "Generic (PLEG): container finished" podID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerID="e3d94ce9b14c0fc83b129ed259b7bc94274fdcf338d612ed25f0ac02f4851575" exitCode=0
Sep 11 12:57:02.871227 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.871183    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf","Type":"ContainerDied","Data":"e3d94ce9b14c0fc83b129ed259b7bc94274fdcf338d612ed25f0ac02f4851575"}
Sep 11 12:57:02.908538 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:02.908488    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7xl7v" podStartSLOduration=3.335853318 podStartE2EDuration="4.908472079s" totalImagesPullingTime="1.572618761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:57:02.907530107 +0000 UTC m=+76.847231442" watchObservedRunningTime="2026-09-11 12:57:02.908472079 +0000 UTC m=+76.848173412"
Sep 11 12:57:03.019432 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.019386    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7875bc4d55-kpt8w"]
Sep 11 12:57:03.024383 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.024357    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.027457 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.027410    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 11 12:57:03.027592 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.027543    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-739vn4fkd7tmd\""
Sep 11 12:57:03.027715 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.027696    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 11 12:57:03.027791 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.027742    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 12:57:03.027791 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.027777    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 11 12:57:03.028089 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.028071    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-xc7rt\""
Sep 11 12:57:03.042395 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.042377    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7875bc4d55-kpt8w"]
Sep 11 12:57:03.105029 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.104999    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0453471e-decd-4d92-9a1d-2af871b1aac6-client-ca-bundle\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.105168 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.105052    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0453471e-decd-4d92-9a1d-2af871b1aac6-metrics-server-audit-profiles\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.105168 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.105092    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0453471e-decd-4d92-9a1d-2af871b1aac6-secret-metrics-server-tls\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.105168 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.105124    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0453471e-decd-4d92-9a1d-2af871b1aac6-secret-metrics-server-client-certs\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.105168 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.105152    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srtpb\" (UniqueName: \"kubernetes.io/projected/0453471e-decd-4d92-9a1d-2af871b1aac6-kube-api-access-srtpb\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.105362 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.105192    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0453471e-decd-4d92-9a1d-2af871b1aac6-audit-log\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.105362 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.105288    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0453471e-decd-4d92-9a1d-2af871b1aac6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.171342 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:57:03.171313    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod126a62b4_8979_4c34_a839_bb18421a90a7.slice/crio-77c724f9ab6d272a6e6e75300180e8fba54e4f5e177cb040097577246b79b148 WatchSource:0}: Error finding container 77c724f9ab6d272a6e6e75300180e8fba54e4f5e177cb040097577246b79b148: Status 404 returned error can't find the container with id 77c724f9ab6d272a6e6e75300180e8fba54e4f5e177cb040097577246b79b148
Sep 11 12:57:03.177341 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.177244    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-bb957f59b-2nsgs"]
Sep 11 12:57:03.206320 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.206295    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0453471e-decd-4d92-9a1d-2af871b1aac6-metrics-server-audit-profiles\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.206413 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.206347    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0453471e-decd-4d92-9a1d-2af871b1aac6-secret-metrics-server-tls\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.206413 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.206382    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0453471e-decd-4d92-9a1d-2af871b1aac6-secret-metrics-server-client-certs\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.206560 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.206413    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srtpb\" (UniqueName: \"kubernetes.io/projected/0453471e-decd-4d92-9a1d-2af871b1aac6-kube-api-access-srtpb\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.206560 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.206467    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0453471e-decd-4d92-9a1d-2af871b1aac6-audit-log\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.206560 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.206505    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0453471e-decd-4d92-9a1d-2af871b1aac6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.206704 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.206572    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0453471e-decd-4d92-9a1d-2af871b1aac6-client-ca-bundle\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.207130 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.207053    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0453471e-decd-4d92-9a1d-2af871b1aac6-audit-log\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.207277 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.207255    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0453471e-decd-4d92-9a1d-2af871b1aac6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.207495 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.207473    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0453471e-decd-4d92-9a1d-2af871b1aac6-metrics-server-audit-profiles\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.209449 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.209374    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0453471e-decd-4d92-9a1d-2af871b1aac6-secret-metrics-server-tls\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.209449 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.209374    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0453471e-decd-4d92-9a1d-2af871b1aac6-client-ca-bundle\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.209864 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.209841    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0453471e-decd-4d92-9a1d-2af871b1aac6-secret-metrics-server-client-certs\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.217974 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.217955    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srtpb\" (UniqueName: \"kubernetes.io/projected/0453471e-decd-4d92-9a1d-2af871b1aac6-kube-api-access-srtpb\") pod \"metrics-server-7875bc4d55-kpt8w\" (UID: \"0453471e-decd-4d92-9a1d-2af871b1aac6\") " pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.335952 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.335919    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:03.632378 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.632235    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7875bc4d55-kpt8w"]
Sep 11 12:57:03.635636 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:57:03.635604    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0453471e_decd_4d92_9a1d_2af871b1aac6.slice/crio-9fc2bbba5b34c1d1d884747d8ea5c48e63a5301b7cc276cc66dc0bcc7e08db0f WatchSource:0}: Error finding container 9fc2bbba5b34c1d1d884747d8ea5c48e63a5301b7cc276cc66dc0bcc7e08db0f: Status 404 returned error can't find the container with id 9fc2bbba5b34c1d1d884747d8ea5c48e63a5301b7cc276cc66dc0bcc7e08db0f
Sep 11 12:57:03.879075 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.879037    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w" event={"ID":"0453471e-decd-4d92-9a1d-2af871b1aac6","Type":"ContainerStarted","Data":"9fc2bbba5b34c1d1d884747d8ea5c48e63a5301b7cc276cc66dc0bcc7e08db0f"}
Sep 11 12:57:03.881613 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:03.881581    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs" event={"ID":"126a62b4-8979-4c34-a839-bb18421a90a7","Type":"ContainerStarted","Data":"77c724f9ab6d272a6e6e75300180e8fba54e4f5e177cb040097577246b79b148"}
Sep 11 12:57:04.897168 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.897010    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf","Type":"ContainerStarted","Data":"44bc48fdbb60b49933bb5a24cfd6caee46c70192d0135174adcd498f7a3efabc"}
Sep 11 12:57:04.897168 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.897133    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf","Type":"ContainerStarted","Data":"5421179c9853728ed6fd0383bcd189a83693385a4e1170f2f63a65b46255832e"}
Sep 11 12:57:04.897692 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.897178    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf","Type":"ContainerStarted","Data":"c5b492d23f0d0255f4193f3f99f7d3918ae49d4daa5547ba87b527cbcbbf5c30"}
Sep 11 12:57:04.897692 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.897219    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf","Type":"ContainerStarted","Data":"aee8a320577134d7c2104afedd7257f4e577dfd41c33ff7cf7e923e357c30351"}
Sep 11 12:57:04.897692 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.897268    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf","Type":"ContainerStarted","Data":"0eece78d50ce7660beb57513299040661335dd35a966b5b580fecf4b6b610f62"}
Sep 11 12:57:04.970621 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.970517    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 12:57:04.977744 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.977663    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:04.981783 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.981727    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 12:57:04.981783 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.981768    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 12:57:04.982079 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.982039    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9e4e3jdrjlev5\""
Sep 11 12:57:04.982158 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.982143    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 12:57:04.985221 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.984165    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 12:57:04.985221 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.984378    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 12:57:04.985221 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.984603    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 12:57:04.985221 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.984766    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 12:57:04.985221 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.984925    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 12:57:04.985221 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.985098    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 12:57:04.986412 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.986300    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pr8vc\""
Sep 11 12:57:04.986412 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.986378    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 12:57:04.987981 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.986675    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 12:57:04.987981 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.987059    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 12:57:04.988715 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.988589    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 12:57:04.990814 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.989910    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 12:57:04.998576 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:04.998551    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 12:57:05.122609 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.122568    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-config-out\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.122609 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.122611    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.122840 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.122660    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.122840 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.122741    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.122840 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.122772    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.122840 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.122814    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.123047 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.122855    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-web-config\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.123047 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.122876    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.123047 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.122900    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.123047 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.122925    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.123047 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.122948    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7ftf\" (UniqueName: \"kubernetes.io/projected/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-kube-api-access-z7ftf\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.123047 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.122986    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.123047 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.123003    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.123047 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.123026    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.123478 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.123083    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.123478 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.123117    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.123478 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.123172    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-config\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.123478 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.123197    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.123478 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.123237    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.123478 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.123271    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224094 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224051    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224246 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224109    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224246 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224142    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224246 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224175    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224246 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224215    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224482 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224261    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-config\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224482 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224287    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224482 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224311    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224482 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224333    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224482 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224360    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-config-out\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224482 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224382    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224482 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224410    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224482 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224471    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224863 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224500    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224863 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224536    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224863 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-web-config\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224863 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224606    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224863 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224640    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224863 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224668    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224863 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224693    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7ftf\" (UniqueName: \"kubernetes.io/projected/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-kube-api-access-z7ftf\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.224863 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.224818    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.226400 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.226371    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.227192 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.227161    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.228500 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.228105    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.239103 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.228943    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.239103 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.229640    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.239103 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.229825    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.239103 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.232499    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.239103 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.236041    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7ftf\" (UniqueName: \"kubernetes.io/projected/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-kube-api-access-z7ftf\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.239103 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.238154    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.240447 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.240303    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.240447 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.240332    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-config\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.240447 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.240398    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.240895 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.240857    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.242831 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.242620    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-config-out\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.242831 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.242774    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-web-config\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.242975 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.242899    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.243290 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.243268    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.244602 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.244507    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.246075 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.245994    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:05.294394 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:05.294095    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:06.730878 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:06.730846    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 12:57:06.913284 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:06.913199    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w" event={"ID":"0453471e-decd-4d92-9a1d-2af871b1aac6","Type":"ContainerStarted","Data":"55be29b781ca297122d7e39cbeacc4966114d441ad396c75f72966608d36d9e9"}
Sep 11 12:57:06.915452 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:06.915268    2582 generic.go:356] "Generic (PLEG): container finished" podID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerID="730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01" exitCode=0
Sep 11 12:57:06.915452 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:06.915354    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4","Type":"ContainerDied","Data":"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01"}
Sep 11 12:57:06.915452 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:06.915398    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4","Type":"ContainerStarted","Data":"2e4b10c168502bcdbca52d1b8d9a582bf049472fbdf7a24faa5be44002c0d990"}
Sep 11 12:57:06.928689 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:06.928661    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs" event={"ID":"126a62b4-8979-4c34-a839-bb18421a90a7","Type":"ContainerStarted","Data":"9f73417910e9e23ce54883475504eada29292c91924de020ea68d32ac583e95e"}
Sep 11 12:57:06.928902 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:06.928878    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs" event={"ID":"126a62b4-8979-4c34-a839-bb18421a90a7","Type":"ContainerStarted","Data":"42d01afac18936f18416f1c5c9a3232b6bef0f341c7428f8fa07e9ba2189c45e"}
Sep 11 12:57:06.928991 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:06.928932    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs" event={"ID":"126a62b4-8979-4c34-a839-bb18421a90a7","Type":"ContainerStarted","Data":"557d71589c5fead1a6ad471db468dff5b48356229fadd67faf47974d4e219e40"}
Sep 11 12:57:06.929065 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:06.928974    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs" event={"ID":"126a62b4-8979-4c34-a839-bb18421a90a7","Type":"ContainerStarted","Data":"6312b5cd48df5b2b5c750606b8fa60a980412f0b071b3565ebf1a32ede8f4b0e"}
Sep 11 12:57:06.929065 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:06.929060    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs" event={"ID":"126a62b4-8979-4c34-a839-bb18421a90a7","Type":"ContainerStarted","Data":"b86b87bc5aa3091bbf8891cc24392a553eab5ef962682d800b101e54aa0157ab"}
Sep 11 12:57:06.933947 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:06.933624    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf","Type":"ContainerStarted","Data":"7a5c32163147ae9f6250ef00f38c9ee496d4cad500b2e741eb0e84be03992cfb"}
Sep 11 12:57:06.959891 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:06.959836    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w" podStartSLOduration=2.175874105 podStartE2EDuration="4.959814274s" totalImagesPullingTime="2.783940169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:57:06.956729975 +0000 UTC m=+80.896431308" watchObservedRunningTime="2026-09-11 12:57:06.959814274 +0000 UTC m=+80.899515608"
Sep 11 12:57:07.050329 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:07.049714    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.487285841 podStartE2EDuration="8.049692921s" totalImagesPullingTime="4.542642845s" totalInitContainerRuntime="19.764235ms" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:57:07.047530096 +0000 UTC m=+80.987231430" watchObservedRunningTime="2026-09-11 12:57:07.049692921 +0000 UTC m=+80.989394259"
Sep 11 12:57:07.944378 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:07.944302    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs" event={"ID":"126a62b4-8979-4c34-a839-bb18421a90a7","Type":"ContainerStarted","Data":"c49c8819bcf2a638e7033358cd765f280d2387bf281e8603c9af420b39496f36"}
Sep 11 12:57:07.993167 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:07.993111    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs" podStartSLOduration=2.747476701 podStartE2EDuration="5.993087667s" totalImagesPullingTime="3.245610966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:57:07.991902325 +0000 UTC m=+81.931603681" watchObservedRunningTime="2026-09-11 12:57:07.993087667 +0000 UTC m=+81.932789002"
Sep 11 12:57:08.950897 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:08.950864    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:09.772474 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:09.772399    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5f7b4f7974-hwkmf"
Sep 11 12:57:14.963515 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:14.963482    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-bb957f59b-2nsgs"
Sep 11 12:57:19.000658 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:19.000620    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-2zvkx" event={"ID":"7a54332c-5c8c-4224-ad6b-e9e4be156923","Type":"ContainerStarted","Data":"e3019e6548e6839662a562febb238dcc0606deb5695fdfd07cfce04458c72b2a"}
Sep 11 12:57:19.001188 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:19.000922    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-2zvkx"
Sep 11 12:57:19.005115 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:19.005090    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4","Type":"ContainerStarted","Data":"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"}
Sep 11 12:57:19.005222 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:19.005149    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4","Type":"ContainerStarted","Data":"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"}
Sep 11 12:57:19.005222 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:19.005191    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4","Type":"ContainerStarted","Data":"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"}
Sep 11 12:57:19.005313 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:19.005224    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4","Type":"ContainerStarted","Data":"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"}
Sep 11 12:57:19.005313 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:19.005251    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4","Type":"ContainerStarted","Data":"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"}
Sep 11 12:57:19.005313 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:19.005289    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4","Type":"ContainerStarted","Data":"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"}
Sep 11 12:57:19.019968 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:19.019943    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-2zvkx"
Sep 11 12:57:19.020842 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:19.020799    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-2zvkx" podStartSLOduration=2.261628369 podStartE2EDuration="21.020785384s" totalImagesPullingTime="18.759157015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:57:19.019117826 +0000 UTC m=+92.958819160" watchObservedRunningTime="2026-09-11 12:57:19.020785384 +0000 UTC m=+92.960486730"
Sep 11 12:57:19.055841 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:19.055783    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.811491657 podStartE2EDuration="15.05576752s" totalImagesPullingTime="11.211501618s" totalInitContainerRuntime="32.774245ms" isStatefulPod=false podCreationTimestamp="2026-09-11 12:57:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:57:19.05371375 +0000 UTC m=+92.993415085" watchObservedRunningTime="2026-09-11 12:57:19.05576752 +0000 UTC m=+92.995468853"
Sep 11 12:57:20.295101 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:20.295053    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:57:23.336135 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:23.336093    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:23.336648 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:23.336168    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:29.843052 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:29.843021    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6k8nb"
Sep 11 12:57:43.342120 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:43.342086    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:57:43.346215 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:57:43.346177    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7875bc4d55-kpt8w"
Sep 11 12:58:05.294593 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:05.294556    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:05.351175 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:05.351148    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:06.227410 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:06.227373    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:19.058816 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:19.058780    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 12:58:19.059509 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:19.059089    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="alertmanager" containerID="cri-o://0eece78d50ce7660beb57513299040661335dd35a966b5b580fecf4b6b610f62" gracePeriod=120
Sep 11 12:58:19.059509 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:19.059144    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="kube-rbac-proxy" containerID="cri-o://5421179c9853728ed6fd0383bcd189a83693385a4e1170f2f63a65b46255832e" gracePeriod=120
Sep 11 12:58:19.059509 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:19.059172    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="kube-rbac-proxy-web" containerID="cri-o://c5b492d23f0d0255f4193f3f99f7d3918ae49d4daa5547ba87b527cbcbbf5c30" gracePeriod=120
Sep 11 12:58:19.059509 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:19.059211    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="config-reloader" containerID="cri-o://aee8a320577134d7c2104afedd7257f4e577dfd41c33ff7cf7e923e357c30351" gracePeriod=120
Sep 11 12:58:19.059509 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:19.059227    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="kube-rbac-proxy-metric" containerID="cri-o://44bc48fdbb60b49933bb5a24cfd6caee46c70192d0135174adcd498f7a3efabc" gracePeriod=120
Sep 11 12:58:19.059509 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:19.059139    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="prom-label-proxy" containerID="cri-o://7a5c32163147ae9f6250ef00f38c9ee496d4cad500b2e741eb0e84be03992cfb" gracePeriod=120
Sep 11 12:58:19.271094 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:19.271061    2582 generic.go:356] "Generic (PLEG): container finished" podID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerID="7a5c32163147ae9f6250ef00f38c9ee496d4cad500b2e741eb0e84be03992cfb" exitCode=0
Sep 11 12:58:19.271094 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:19.271083    2582 generic.go:356] "Generic (PLEG): container finished" podID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerID="5421179c9853728ed6fd0383bcd189a83693385a4e1170f2f63a65b46255832e" exitCode=0
Sep 11 12:58:19.271094 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:19.271091    2582 generic.go:356] "Generic (PLEG): container finished" podID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerID="aee8a320577134d7c2104afedd7257f4e577dfd41c33ff7cf7e923e357c30351" exitCode=0
Sep 11 12:58:19.271094 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:19.271098    2582 generic.go:356] "Generic (PLEG): container finished" podID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerID="0eece78d50ce7660beb57513299040661335dd35a966b5b580fecf4b6b610f62" exitCode=0
Sep 11 12:58:19.271286 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:19.271135    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf","Type":"ContainerDied","Data":"7a5c32163147ae9f6250ef00f38c9ee496d4cad500b2e741eb0e84be03992cfb"}
Sep 11 12:58:19.271286 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:19.271197    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf","Type":"ContainerDied","Data":"5421179c9853728ed6fd0383bcd189a83693385a4e1170f2f63a65b46255832e"}
Sep 11 12:58:19.271286 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:19.271224    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf","Type":"ContainerDied","Data":"aee8a320577134d7c2104afedd7257f4e577dfd41c33ff7cf7e923e357c30351"}
Sep 11 12:58:19.271286 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:19.271250    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf","Type":"ContainerDied","Data":"0eece78d50ce7660beb57513299040661335dd35a966b5b580fecf4b6b610f62"}
Sep 11 12:58:20.279250 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.279222    2582 generic.go:356] "Generic (PLEG): container finished" podID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerID="44bc48fdbb60b49933bb5a24cfd6caee46c70192d0135174adcd498f7a3efabc" exitCode=0
Sep 11 12:58:20.279250 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.279247    2582 generic.go:356] "Generic (PLEG): container finished" podID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerID="c5b492d23f0d0255f4193f3f99f7d3918ae49d4daa5547ba87b527cbcbbf5c30" exitCode=0
Sep 11 12:58:20.279621 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.279293    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf","Type":"ContainerDied","Data":"44bc48fdbb60b49933bb5a24cfd6caee46c70192d0135174adcd498f7a3efabc"}
Sep 11 12:58:20.279621 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.279361    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf","Type":"ContainerDied","Data":"c5b492d23f0d0255f4193f3f99f7d3918ae49d4daa5547ba87b527cbcbbf5c30"}
Sep 11 12:58:20.317594 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.317574    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:20.373840 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.373774    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy-web\") pod \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") "
Sep 11 12:58:20.373840 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.373826    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-main-tls\") pod \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") "
Sep 11 12:58:20.374011 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.373859    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-tls-assets\") pod \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") "
Sep 11 12:58:20.374011 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.373889    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy\") pod \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") "
Sep 11 12:58:20.374011 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.373906    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") "
Sep 11 12:58:20.374011 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.373945    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdgd7\" (UniqueName: \"kubernetes.io/projected/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-kube-api-access-jdgd7\") pod \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") "
Sep 11 12:58:20.374011 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.373972    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-config-volume\") pod \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") "
Sep 11 12:58:20.374248 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.374067    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-alertmanager-trusted-ca-bundle\") pod \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") "
Sep 11 12:58:20.374248 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.374112    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-web-config\") pod \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") "
Sep 11 12:58:20.374248 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.374161    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-alertmanager-main-db\") pod \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") "
Sep 11 12:58:20.374248 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.374191    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-metrics-client-ca\") pod \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") "
Sep 11 12:58:20.374248 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.374222    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-config-out\") pod \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") "
Sep 11 12:58:20.374521 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.374253    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-cluster-tls-config\") pod \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\" (UID: \"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf\") "
Sep 11 12:58:20.374992 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.374961    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" (UID: "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:58:20.375223 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.375192    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" (UID: "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:58:20.375695 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.375669    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" (UID: "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:58:20.377072 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.377027    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-kube-api-access-jdgd7" (OuterVolumeSpecName: "kube-api-access-jdgd7") pod "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" (UID: "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf"). InnerVolumeSpecName "kube-api-access-jdgd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:58:20.377953 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.377896    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" (UID: "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:20.378191 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.378143    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" (UID: "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:20.378588 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.378550    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" (UID: "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:20.379063 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.379023    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" (UID: "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:20.379063 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.379037    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-config-volume" (OuterVolumeSpecName: "config-volume") pod "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" (UID: "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:20.379063 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.379041    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" (UID: "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:58:20.379063 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.379046    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-config-out" (OuterVolumeSpecName: "config-out") pod "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" (UID: "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:58:20.383222 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.383193    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" (UID: "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:20.390211 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.390186    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-web-config" (OuterVolumeSpecName: "web-config") pod "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" (UID: "09ed0d59-d217-405b-9c84-e7b7ac2dc1bf"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:20.475035 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.475008    2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-config-out\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:20.475114 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.475036    2582 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-cluster-tls-config\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:20.475114 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.475052    2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:20.475114 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.475068    2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-main-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:20.475114 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.475083    2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-tls-assets\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:20.475114 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.475097    2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:20.475114 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.475111    2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:20.475306 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.475126    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdgd7\" (UniqueName: \"kubernetes.io/projected/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-kube-api-access-jdgd7\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:20.475306 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.475152    2582 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-config-volume\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:20.475306 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.475167    2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:20.475306 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.475180    2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-web-config\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:20.475306 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.475193    2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-alertmanager-main-db\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:20.475306 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:20.475206    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf-metrics-client-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:21.286502 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.286457    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09ed0d59-d217-405b-9c84-e7b7ac2dc1bf","Type":"ContainerDied","Data":"b48300d5983707afe8cab24e131366c5f2c5bdec34a49f1376024845134c5b3c"}
Sep 11 12:58:21.286502 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.286494    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.286911 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.286569    2582 scope.go:117] "RemoveContainer" containerID="7a5c32163147ae9f6250ef00f38c9ee496d4cad500b2e741eb0e84be03992cfb"
Sep 11 12:58:21.299800 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.299776    2582 scope.go:117] "RemoveContainer" containerID="44bc48fdbb60b49933bb5a24cfd6caee46c70192d0135174adcd498f7a3efabc"
Sep 11 12:58:21.307850 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.307831    2582 scope.go:117] "RemoveContainer" containerID="5421179c9853728ed6fd0383bcd189a83693385a4e1170f2f63a65b46255832e"
Sep 11 12:58:21.312735 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.312709    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 12:58:21.316033 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.316002    2582 scope.go:117] "RemoveContainer" containerID="c5b492d23f0d0255f4193f3f99f7d3918ae49d4daa5547ba87b527cbcbbf5c30"
Sep 11 12:58:21.318474 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.318450    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 12:58:21.323176 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.323161    2582 scope.go:117] "RemoveContainer" containerID="aee8a320577134d7c2104afedd7257f4e577dfd41c33ff7cf7e923e357c30351"
Sep 11 12:58:21.329731 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.329715    2582 scope.go:117] "RemoveContainer" containerID="0eece78d50ce7660beb57513299040661335dd35a966b5b580fecf4b6b610f62"
Sep 11 12:58:21.336724 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.336706    2582 scope.go:117] "RemoveContainer" containerID="e3d94ce9b14c0fc83b129ed259b7bc94274fdcf338d612ed25f0ac02f4851575"
Sep 11 12:58:21.345958 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.345938    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 12:58:21.346256 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346244    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="config-reloader"
Sep 11 12:58:21.346296 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346258    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="config-reloader"
Sep 11 12:58:21.346296 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346270    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="init-config-reloader"
Sep 11 12:58:21.346296 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346276    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="init-config-reloader"
Sep 11 12:58:21.346296 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346283    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="kube-rbac-proxy-metric"
Sep 11 12:58:21.346296 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346288    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="kube-rbac-proxy-metric"
Sep 11 12:58:21.346296 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346295    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="kube-rbac-proxy"
Sep 11 12:58:21.346510 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346300    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="kube-rbac-proxy"
Sep 11 12:58:21.346510 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346309    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="prom-label-proxy"
Sep 11 12:58:21.346510 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346314    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="prom-label-proxy"
Sep 11 12:58:21.346510 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346323    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="kube-rbac-proxy-web"
Sep 11 12:58:21.346510 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346328    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="kube-rbac-proxy-web"
Sep 11 12:58:21.346510 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346340    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="alertmanager"
Sep 11 12:58:21.346510 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346346    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="alertmanager"
Sep 11 12:58:21.346510 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346390    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="kube-rbac-proxy"
Sep 11 12:58:21.346510 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346396    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="kube-rbac-proxy-metric"
Sep 11 12:58:21.346510 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346404    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="prom-label-proxy"
Sep 11 12:58:21.346510 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346410    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="alertmanager"
Sep 11 12:58:21.346510 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346432    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="config-reloader"
Sep 11 12:58:21.346510 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.346439    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" containerName="kube-rbac-proxy-web"
Sep 11 12:58:21.351159 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.351141    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.353709 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.353691    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 12:58:21.353804 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.353718    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 12:58:21.353936 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.353868    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rzdbn\""
Sep 11 12:58:21.353936 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.353899    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 12:58:21.353936 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.353915    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 12:58:21.354111 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.353986    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 12:58:21.354186 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.354159    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 12:58:21.354249 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.354195    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 12:58:21.354249 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.354210    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 12:58:21.360481 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.360415    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 12:58:21.364707 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.364690    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 12:58:21.382565 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.382539    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-config-volume\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.382646 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.382573    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftp9l\" (UniqueName: \"kubernetes.io/projected/0a2a14b2-8b2e-4647-bac0-0346c745e357-kube-api-access-ftp9l\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.382646 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.382592    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.382646 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.382609    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a2a14b2-8b2e-4647-bac0-0346c745e357-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.382753 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.382686    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-web-config\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.382753 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.382722    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a2a14b2-8b2e-4647-bac0-0346c745e357-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.382827 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.382752    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.382827 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.382768    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a2a14b2-8b2e-4647-bac0-0346c745e357-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.382892 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.382827    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.382892 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.382856    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.382955 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.382904    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.382955 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.382919    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2a14b2-8b2e-4647-bac0-0346c745e357-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.382955 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.382940    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a2a14b2-8b2e-4647-bac0-0346c745e357-config-out\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.483747 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.483724    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.483831 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.483768    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.483831 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.483799    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.483831 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.483822    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2a14b2-8b2e-4647-bac0-0346c745e357-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.483981 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.483863    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a2a14b2-8b2e-4647-bac0-0346c745e357-config-out\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.483981 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.483898    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-config-volume\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.483981 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.483933    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftp9l\" (UniqueName: \"kubernetes.io/projected/0a2a14b2-8b2e-4647-bac0-0346c745e357-kube-api-access-ftp9l\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.483981 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.483958    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.484163 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.484000    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a2a14b2-8b2e-4647-bac0-0346c745e357-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.484163 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.484045    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-web-config\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.484163 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.484069    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a2a14b2-8b2e-4647-bac0-0346c745e357-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.484163 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.484104    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.484163 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.484130    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a2a14b2-8b2e-4647-bac0-0346c745e357-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.484859 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.484833    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a2a14b2-8b2e-4647-bac0-0346c745e357-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.484963 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.484843    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a2a14b2-8b2e-4647-bac0-0346c745e357-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.485126 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.485100    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a2a14b2-8b2e-4647-bac0-0346c745e357-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.486908 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.486885    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.487106 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.487075    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.487106 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.487094    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.487344 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.487323    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a2a14b2-8b2e-4647-bac0-0346c745e357-config-out\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.487500 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.487472    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.487654 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.487632    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-web-config\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.487825 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.487811    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.488187 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.488168    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a2a14b2-8b2e-4647-bac0-0346c745e357-config-volume\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.488807 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.488789    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a2a14b2-8b2e-4647-bac0-0346c745e357-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.492984 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.492963    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftp9l\" (UniqueName: \"kubernetes.io/projected/0a2a14b2-8b2e-4647-bac0-0346c745e357-kube-api-access-ftp9l\") pod \"alertmanager-main-0\" (UID: \"0a2a14b2-8b2e-4647-bac0-0346c745e357\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.661437 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.661373    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 12:58:21.947728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:21.947704    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 12:58:21.950235 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:58:21.950208    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a2a14b2_8b2e_4647_bac0_0346c745e357.slice/crio-57ee006b88cb27f8bc446cdf86f0d2d6d9de020b95bc9236f43a0ed826c8ecd9 WatchSource:0}: Error finding container 57ee006b88cb27f8bc446cdf86f0d2d6d9de020b95bc9236f43a0ed826c8ecd9: Status 404 returned error can't find the container with id 57ee006b88cb27f8bc446cdf86f0d2d6d9de020b95bc9236f43a0ed826c8ecd9
Sep 11 12:58:22.292499 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:22.292467    2582 generic.go:356] "Generic (PLEG): container finished" podID="0a2a14b2-8b2e-4647-bac0-0346c745e357" containerID="abeb645294a6eda8faffff65b6d3eb64cf5f3c8585a3df942cf0ab2226a534a2" exitCode=0
Sep 11 12:58:22.292851 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:22.292562    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a2a14b2-8b2e-4647-bac0-0346c745e357","Type":"ContainerDied","Data":"abeb645294a6eda8faffff65b6d3eb64cf5f3c8585a3df942cf0ab2226a534a2"}
Sep 11 12:58:22.292851 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:22.292638    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a2a14b2-8b2e-4647-bac0-0346c745e357","Type":"ContainerStarted","Data":"57ee006b88cb27f8bc446cdf86f0d2d6d9de020b95bc9236f43a0ed826c8ecd9"}
Sep 11 12:58:22.476125 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:22.476097    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09ed0d59-d217-405b-9c84-e7b7ac2dc1bf" path="/var/lib/kubelet/pods/09ed0d59-d217-405b-9c84-e7b7ac2dc1bf/volumes"
Sep 11 12:58:23.303143 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.303106    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a2a14b2-8b2e-4647-bac0-0346c745e357","Type":"ContainerStarted","Data":"6f8a887b8eec1ce44b9f0d028a9a178e4eacdc32e74189cd6ccfad8c0b0bce30"}
Sep 11 12:58:23.303631 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.303197    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a2a14b2-8b2e-4647-bac0-0346c745e357","Type":"ContainerStarted","Data":"c41e5b85e613db04d2bd2761bf9396b0f055be1a63d73d071bd9f79b1b1375b8"}
Sep 11 12:58:23.303631 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.303225    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a2a14b2-8b2e-4647-bac0-0346c745e357","Type":"ContainerStarted","Data":"b1e3d48ca58e70bd08ffac39b5ed118b76a4947699eee297508bc716d4b5391c"}
Sep 11 12:58:23.303631 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.303249    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a2a14b2-8b2e-4647-bac0-0346c745e357","Type":"ContainerStarted","Data":"f22ba8e2102b55705deac4e4a48e9d82aeb39a73fba5c825e6a582a66673c5ee"}
Sep 11 12:58:23.303631 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.303272    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a2a14b2-8b2e-4647-bac0-0346c745e357","Type":"ContainerStarted","Data":"943303be91f363a3e6657ad33d90618ddd5eed28ab3b404128c9148edef3bc96"}
Sep 11 12:58:23.303631 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.303295    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a2a14b2-8b2e-4647-bac0-0346c745e357","Type":"ContainerStarted","Data":"ccddd377076eeb9eb17aa714ed22d5a98d44aa8f548dd76257a12d180b62efa5"}
Sep 11 12:58:23.313413 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.313373    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 12:58:23.313981 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.313946    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="prometheus" containerID="cri-o://36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a" gracePeriod=600
Sep 11 12:58:23.314247 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.313956    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="thanos-sidecar" containerID="cri-o://2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2" gracePeriod=600
Sep 11 12:58:23.314352 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.313956    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="kube-rbac-proxy" containerID="cri-o://32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c" gracePeriod=600
Sep 11 12:58:23.314408 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.314009    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="kube-rbac-proxy-web" containerID="cri-o://b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d" gracePeriod=600
Sep 11 12:58:23.314503 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.314011    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="kube-rbac-proxy-thanos" containerID="cri-o://6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267" gracePeriod=600
Sep 11 12:58:23.314924 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.314009    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="config-reloader" containerID="cri-o://567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58" gracePeriod=600
Sep 11 12:58:23.345061 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.344972    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.326227958 podStartE2EDuration="2.344952569s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.724611ms" isStatefulPod=false podCreationTimestamp="2026-09-11 12:58:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:58:23.341371879 +0000 UTC m=+157.281073227" watchObservedRunningTime="2026-09-11 12:58:23.344952569 +0000 UTC m=+157.284653901"
Sep 11 12:58:23.555856 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.555522    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603215    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-2\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603261    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7ftf\" (UniqueName: \"kubernetes.io/projected/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-kube-api-access-z7ftf\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603292    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-config-out\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603327    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-thanos-prometheus-http-client-file\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603353    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-serving-certs-ca-bundle\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603381    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-tls-assets\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603412    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603464    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-kube-rbac-proxy\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603519    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-metrics-client-ca\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603541    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-config\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603572    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603612    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-kubelet-serving-ca-bundle\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603645    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-tls\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603689    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-web-config\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603718    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-db\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603751    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-metrics-client-certs\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603743    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:58:23.606235 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603791    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-grpc-tls\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.608013 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603817    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-0\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.608013 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603840    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-1\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.608013 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603872    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-trusted-ca-bundle\") pod \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\" (UID: \"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4\") "
Sep 11 12:58:23.608013 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.603998    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:58:23.608013 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.604209    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.608013 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.604227    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.608013 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.604343    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:58:23.608013 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.604722    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:58:23.608013 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.605911    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:58:23.608013 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.606606    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:58:23.608013 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.607967    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:58:23.609499 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.608715    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-config-out" (OuterVolumeSpecName: "config-out") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:58:23.609499 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.609211    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:23.609499 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.609332    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:23.609499 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.609492    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-config" (OuterVolumeSpecName: "config") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:23.609756 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.609635    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:23.609756 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.609670    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:23.609912 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.609813    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:58:23.609912 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.609813    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-kube-api-access-z7ftf" (OuterVolumeSpecName: "kube-api-access-z7ftf") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "kube-api-access-z7ftf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:58:23.609912 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.609862    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:23.610205 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.610176    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:58:23.610589 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.610554    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:23.611597 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.611561    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:23.622393 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.622370    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-web-config" (OuterVolumeSpecName: "web-config") pod "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" (UID: "68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:58:23.705452 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705411    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705452 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705448    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705461    2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-web-config\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705472    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-db\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705480    2582 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-metrics-client-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705489    2582 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-grpc-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705497    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705505    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705514    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705522    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7ftf\" (UniqueName: \"kubernetes.io/projected/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-kube-api-access-z7ftf\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705529    2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-config-out\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705539    2582 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705546    2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-tls-assets\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705556    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705573    2582 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-kube-rbac-proxy\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705581    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-configmap-metrics-client-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705589    2582 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-config\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:23.705610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:23.705597    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 12:58:24.311560 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.311529    2582 generic.go:356] "Generic (PLEG): container finished" podID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerID="6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267" exitCode=0
Sep 11 12:58:24.311560 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.311556    2582 generic.go:356] "Generic (PLEG): container finished" podID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerID="32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c" exitCode=0
Sep 11 12:58:24.311560 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.311563    2582 generic.go:356] "Generic (PLEG): container finished" podID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerID="b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d" exitCode=0
Sep 11 12:58:24.311956 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.311569    2582 generic.go:356] "Generic (PLEG): container finished" podID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerID="2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2" exitCode=0
Sep 11 12:58:24.311956 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.311576    2582 generic.go:356] "Generic (PLEG): container finished" podID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerID="567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58" exitCode=0
Sep 11 12:58:24.311956 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.311582    2582 generic.go:356] "Generic (PLEG): container finished" podID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerID="36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a" exitCode=0
Sep 11 12:58:24.311956 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.311615    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4","Type":"ContainerDied","Data":"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"}
Sep 11 12:58:24.311956 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.311698    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4","Type":"ContainerDied","Data":"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"}
Sep 11 12:58:24.311956 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.311624    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.311956 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.311728    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4","Type":"ContainerDied","Data":"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"}
Sep 11 12:58:24.311956 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.311755    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4","Type":"ContainerDied","Data":"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"}
Sep 11 12:58:24.311956 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.311780    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4","Type":"ContainerDied","Data":"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"}
Sep 11 12:58:24.311956 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.311794    2582 scope.go:117] "RemoveContainer" containerID="6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"
Sep 11 12:58:24.311956 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.311805    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4","Type":"ContainerDied","Data":"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"}
Sep 11 12:58:24.311956 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.311830    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4","Type":"ContainerDied","Data":"2e4b10c168502bcdbca52d1b8d9a582bf049472fbdf7a24faa5be44002c0d990"}
Sep 11 12:58:24.321054 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.320880    2582 scope.go:117] "RemoveContainer" containerID="32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"
Sep 11 12:58:24.328482 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.328464    2582 scope.go:117] "RemoveContainer" containerID="b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"
Sep 11 12:58:24.335586 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.335559    2582 scope.go:117] "RemoveContainer" containerID="2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"
Sep 11 12:58:24.338362 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.338334    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 12:58:24.342647 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.342627    2582 scope.go:117] "RemoveContainer" containerID="567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"
Sep 11 12:58:24.344923 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.344904    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 12:58:24.349722 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.349706    2582 scope.go:117] "RemoveContainer" containerID="36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"
Sep 11 12:58:24.356726 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.356708    2582 scope.go:117] "RemoveContainer" containerID="730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01"
Sep 11 12:58:24.363222 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.363212    2582 scope.go:117] "RemoveContainer" containerID="6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"
Sep 11 12:58:24.363531 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:58:24.363510    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267\": container with ID starting with 6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267 not found: ID does not exist" containerID="6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"
Sep 11 12:58:24.363612 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.363544    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"} err="failed to get container status \"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267\": rpc error: code = NotFound desc = could not find container \"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267\": container with ID starting with 6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267 not found: ID does not exist"
Sep 11 12:58:24.363612 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.363588    2582 scope.go:117] "RemoveContainer" containerID="32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"
Sep 11 12:58:24.363871 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:58:24.363853    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c\": container with ID starting with 32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c not found: ID does not exist" containerID="32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"
Sep 11 12:58:24.363910 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.363884    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"} err="failed to get container status \"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c\": rpc error: code = NotFound desc = could not find container \"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c\": container with ID starting with 32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c not found: ID does not exist"
Sep 11 12:58:24.363910 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.363899    2582 scope.go:117] "RemoveContainer" containerID="b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"
Sep 11 12:58:24.364159 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:58:24.364142    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d\": container with ID starting with b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d not found: ID does not exist" containerID="b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"
Sep 11 12:58:24.364224 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.364167    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"} err="failed to get container status \"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d\": rpc error: code = NotFound desc = could not find container \"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d\": container with ID starting with b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d not found: ID does not exist"
Sep 11 12:58:24.364224 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.364189    2582 scope.go:117] "RemoveContainer" containerID="2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"
Sep 11 12:58:24.364446 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:58:24.364414    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2\": container with ID starting with 2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2 not found: ID does not exist" containerID="2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"
Sep 11 12:58:24.364505 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.364454    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"} err="failed to get container status \"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2\": rpc error: code = NotFound desc = could not find container \"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2\": container with ID starting with 2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2 not found: ID does not exist"
Sep 11 12:58:24.364505 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.364474    2582 scope.go:117] "RemoveContainer" containerID="567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"
Sep 11 12:58:24.364719 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:58:24.364701    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58\": container with ID starting with 567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58 not found: ID does not exist" containerID="567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"
Sep 11 12:58:24.364769 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.364725    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"} err="failed to get container status \"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58\": rpc error: code = NotFound desc = could not find container \"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58\": container with ID starting with 567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58 not found: ID does not exist"
Sep 11 12:58:24.364769 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.364739    2582 scope.go:117] "RemoveContainer" containerID="36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"
Sep 11 12:58:24.364953 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:58:24.364934    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a\": container with ID starting with 36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a not found: ID does not exist" containerID="36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"
Sep 11 12:58:24.365009 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.364959    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"} err="failed to get container status \"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a\": rpc error: code = NotFound desc = could not find container \"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a\": container with ID starting with 36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a not found: ID does not exist"
Sep 11 12:58:24.365009 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.364977    2582 scope.go:117] "RemoveContainer" containerID="730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01"
Sep 11 12:58:24.365220 ip-10-0-143-6 kubenswrapper[2582]: E0911 12:58:24.365197    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01\": container with ID starting with 730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01 not found: ID does not exist" containerID="730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01"
Sep 11 12:58:24.365254 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.365229    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01"} err="failed to get container status \"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01\": rpc error: code = NotFound desc = could not find container \"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01\": container with ID starting with 730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01 not found: ID does not exist"
Sep 11 12:58:24.365254 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.365250    2582 scope.go:117] "RemoveContainer" containerID="6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"
Sep 11 12:58:24.365586 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.365558    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"} err="failed to get container status \"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267\": rpc error: code = NotFound desc = could not find container \"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267\": container with ID starting with 6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267 not found: ID does not exist"
Sep 11 12:58:24.365707 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.365588    2582 scope.go:117] "RemoveContainer" containerID="32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"
Sep 11 12:58:24.365891 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.365862    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"} err="failed to get container status \"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c\": rpc error: code = NotFound desc = could not find container \"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c\": container with ID starting with 32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c not found: ID does not exist"
Sep 11 12:58:24.365891 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.365884    2582 scope.go:117] "RemoveContainer" containerID="b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"
Sep 11 12:58:24.366119 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.366104    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"} err="failed to get container status \"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d\": rpc error: code = NotFound desc = could not find container \"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d\": container with ID starting with b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d not found: ID does not exist"
Sep 11 12:58:24.366165 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.366120    2582 scope.go:117] "RemoveContainer" containerID="2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"
Sep 11 12:58:24.366320 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.366303    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"} err="failed to get container status \"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2\": rpc error: code = NotFound desc = could not find container \"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2\": container with ID starting with 2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2 not found: ID does not exist"
Sep 11 12:58:24.366363 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.366321    2582 scope.go:117] "RemoveContainer" containerID="567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"
Sep 11 12:58:24.366587 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.366564    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"} err="failed to get container status \"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58\": rpc error: code = NotFound desc = could not find container \"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58\": container with ID starting with 567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58 not found: ID does not exist"
Sep 11 12:58:24.366663 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.366589    2582 scope.go:117] "RemoveContainer" containerID="36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"
Sep 11 12:58:24.366820 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.366803    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"} err="failed to get container status \"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a\": rpc error: code = NotFound desc = could not find container \"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a\": container with ID starting with 36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a not found: ID does not exist"
Sep 11 12:58:24.366867 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.366820    2582 scope.go:117] "RemoveContainer" containerID="730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01"
Sep 11 12:58:24.367033 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.367013    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01"} err="failed to get container status \"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01\": rpc error: code = NotFound desc = could not find container \"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01\": container with ID starting with 730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01 not found: ID does not exist"
Sep 11 12:58:24.367103 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.367034    2582 scope.go:117] "RemoveContainer" containerID="6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"
Sep 11 12:58:24.367237 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.367218    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"} err="failed to get container status \"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267\": rpc error: code = NotFound desc = could not find container \"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267\": container with ID starting with 6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267 not found: ID does not exist"
Sep 11 12:58:24.367279 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.367238    2582 scope.go:117] "RemoveContainer" containerID="32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"
Sep 11 12:58:24.367461 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.367443    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"} err="failed to get container status \"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c\": rpc error: code = NotFound desc = could not find container \"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c\": container with ID starting with 32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c not found: ID does not exist"
Sep 11 12:58:24.367532 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.367462    2582 scope.go:117] "RemoveContainer" containerID="b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"
Sep 11 12:58:24.367688 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.367673    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"} err="failed to get container status \"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d\": rpc error: code = NotFound desc = could not find container \"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d\": container with ID starting with b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d not found: ID does not exist"
Sep 11 12:58:24.367733 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.367688    2582 scope.go:117] "RemoveContainer" containerID="2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"
Sep 11 12:58:24.367885 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.367867    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"} err="failed to get container status \"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2\": rpc error: code = NotFound desc = could not find container \"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2\": container with ID starting with 2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2 not found: ID does not exist"
Sep 11 12:58:24.367945 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.367886    2582 scope.go:117] "RemoveContainer" containerID="567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"
Sep 11 12:58:24.368099 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.368084    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"} err="failed to get container status \"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58\": rpc error: code = NotFound desc = could not find container \"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58\": container with ID starting with 567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58 not found: ID does not exist"
Sep 11 12:58:24.368147 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.368099    2582 scope.go:117] "RemoveContainer" containerID="36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"
Sep 11 12:58:24.368284 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.368266    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"} err="failed to get container status \"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a\": rpc error: code = NotFound desc = could not find container \"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a\": container with ID starting with 36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a not found: ID does not exist"
Sep 11 12:58:24.368341 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.368285    2582 scope.go:117] "RemoveContainer" containerID="730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01"
Sep 11 12:58:24.368529 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.368510    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01"} err="failed to get container status \"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01\": rpc error: code = NotFound desc = could not find container \"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01\": container with ID starting with 730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01 not found: ID does not exist"
Sep 11 12:58:24.368600 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.368532    2582 scope.go:117] "RemoveContainer" containerID="6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"
Sep 11 12:58:24.368753 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.368733    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"} err="failed to get container status \"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267\": rpc error: code = NotFound desc = could not find container \"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267\": container with ID starting with 6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267 not found: ID does not exist"
Sep 11 12:58:24.368805 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.368754    2582 scope.go:117] "RemoveContainer" containerID="32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"
Sep 11 12:58:24.368951 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.368934    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"} err="failed to get container status \"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c\": rpc error: code = NotFound desc = could not find container \"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c\": container with ID starting with 32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c not found: ID does not exist"
Sep 11 12:58:24.368991 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.368951    2582 scope.go:117] "RemoveContainer" containerID="b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"
Sep 11 12:58:24.369166 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.369151    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"} err="failed to get container status \"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d\": rpc error: code = NotFound desc = could not find container \"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d\": container with ID starting with b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d not found: ID does not exist"
Sep 11 12:58:24.369166 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.369165    2582 scope.go:117] "RemoveContainer" containerID="2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"
Sep 11 12:58:24.369388 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.369369    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"} err="failed to get container status \"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2\": rpc error: code = NotFound desc = could not find container \"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2\": container with ID starting with 2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2 not found: ID does not exist"
Sep 11 12:58:24.369526 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.369389    2582 scope.go:117] "RemoveContainer" containerID="567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"
Sep 11 12:58:24.369672 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.369655    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"} err="failed to get container status \"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58\": rpc error: code = NotFound desc = could not find container \"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58\": container with ID starting with 567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58 not found: ID does not exist"
Sep 11 12:58:24.369739 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.369673    2582 scope.go:117] "RemoveContainer" containerID="36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"
Sep 11 12:58:24.369911 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.369893    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"} err="failed to get container status \"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a\": rpc error: code = NotFound desc = could not find container \"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a\": container with ID starting with 36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a not found: ID does not exist"
Sep 11 12:58:24.369951 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.369912    2582 scope.go:117] "RemoveContainer" containerID="730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01"
Sep 11 12:58:24.370134 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.370118    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01"} err="failed to get container status \"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01\": rpc error: code = NotFound desc = could not find container \"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01\": container with ID starting with 730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01 not found: ID does not exist"
Sep 11 12:58:24.370194 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.370135    2582 scope.go:117] "RemoveContainer" containerID="6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"
Sep 11 12:58:24.370369 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.370349    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"} err="failed to get container status \"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267\": rpc error: code = NotFound desc = could not find container \"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267\": container with ID starting with 6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267 not found: ID does not exist"
Sep 11 12:58:24.370415 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.370370    2582 scope.go:117] "RemoveContainer" containerID="32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"
Sep 11 12:58:24.370610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.370592    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"} err="failed to get container status \"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c\": rpc error: code = NotFound desc = could not find container \"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c\": container with ID starting with 32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c not found: ID does not exist"
Sep 11 12:58:24.370671 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.370612    2582 scope.go:117] "RemoveContainer" containerID="b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"
Sep 11 12:58:24.370831 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.370814    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"} err="failed to get container status \"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d\": rpc error: code = NotFound desc = could not find container \"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d\": container with ID starting with b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d not found: ID does not exist"
Sep 11 12:58:24.370872 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.370832    2582 scope.go:117] "RemoveContainer" containerID="2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"
Sep 11 12:58:24.371016 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.370998    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"} err="failed to get container status \"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2\": rpc error: code = NotFound desc = could not find container \"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2\": container with ID starting with 2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2 not found: ID does not exist"
Sep 11 12:58:24.371058 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.371017    2582 scope.go:117] "RemoveContainer" containerID="567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"
Sep 11 12:58:24.371218 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.371200    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"} err="failed to get container status \"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58\": rpc error: code = NotFound desc = could not find container \"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58\": container with ID starting with 567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58 not found: ID does not exist"
Sep 11 12:58:24.371287 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.371219    2582 scope.go:117] "RemoveContainer" containerID="36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"
Sep 11 12:58:24.371459 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.371444    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"} err="failed to get container status \"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a\": rpc error: code = NotFound desc = could not find container \"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a\": container with ID starting with 36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a not found: ID does not exist"
Sep 11 12:58:24.371503 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.371459    2582 scope.go:117] "RemoveContainer" containerID="730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01"
Sep 11 12:58:24.371660 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.371642    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01"} err="failed to get container status \"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01\": rpc error: code = NotFound desc = could not find container \"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01\": container with ID starting with 730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01 not found: ID does not exist"
Sep 11 12:58:24.371726 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.371661    2582 scope.go:117] "RemoveContainer" containerID="6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"
Sep 11 12:58:24.371930 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.371910    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267"} err="failed to get container status \"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267\": rpc error: code = NotFound desc = could not find container \"6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267\": container with ID starting with 6a3b176d1a9b4a61cac596a515f521ec68cd510d4a91577f461f8db27f71b267 not found: ID does not exist"
Sep 11 12:58:24.372000 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.371931    2582 scope.go:117] "RemoveContainer" containerID="32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"
Sep 11 12:58:24.372155 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.372135    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c"} err="failed to get container status \"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c\": rpc error: code = NotFound desc = could not find container \"32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c\": container with ID starting with 32b041ccc5eb15a0d03427c220f5782b965ff794fa0c63e0fc6cda6dfd8ad34c not found: ID does not exist"
Sep 11 12:58:24.372202 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.372156    2582 scope.go:117] "RemoveContainer" containerID="b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"
Sep 11 12:58:24.372382 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.372364    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d"} err="failed to get container status \"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d\": rpc error: code = NotFound desc = could not find container \"b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d\": container with ID starting with b3e01378446bf3522fe6fd46e32a1a58d18280689d455f03b247860c3ac7eb4d not found: ID does not exist"
Sep 11 12:58:24.372472 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.372384    2582 scope.go:117] "RemoveContainer" containerID="2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"
Sep 11 12:58:24.372657 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.372639    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2"} err="failed to get container status \"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2\": rpc error: code = NotFound desc = could not find container \"2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2\": container with ID starting with 2a1883133bd07d4bfeb8a8e91210c96a3fad5b3cb7b5e283122d4b0e003139b2 not found: ID does not exist"
Sep 11 12:58:24.372728 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.372658    2582 scope.go:117] "RemoveContainer" containerID="567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"
Sep 11 12:58:24.372906 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.372885    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58"} err="failed to get container status \"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58\": rpc error: code = NotFound desc = could not find container \"567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58\": container with ID starting with 567fb1de2f257c0209141e00ef0808e65dcc769f59c651de2d1970f9fd18ba58 not found: ID does not exist"
Sep 11 12:58:24.372947 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.372907    2582 scope.go:117] "RemoveContainer" containerID="36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"
Sep 11 12:58:24.373125 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.373108    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a"} err="failed to get container status \"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a\": rpc error: code = NotFound desc = could not find container \"36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a\": container with ID starting with 36df7006b3a4a9b1d060c60bb1773b38f37963c22898995a35f406230c583a4a not found: ID does not exist"
Sep 11 12:58:24.373184 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.373126    2582 scope.go:117] "RemoveContainer" containerID="730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01"
Sep 11 12:58:24.373366 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.373349    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01"} err="failed to get container status \"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01\": rpc error: code = NotFound desc = could not find container \"730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01\": container with ID starting with 730579b17d4affc69b56e1afbcb23317d695f18325b1dc1a12c4359d9332af01 not found: ID does not exist"
Sep 11 12:58:24.378878 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.378856    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 12:58:24.379185 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379169    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="prometheus"
Sep 11 12:58:24.379263 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379188    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="prometheus"
Sep 11 12:58:24.379263 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379201    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="config-reloader"
Sep 11 12:58:24.379263 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379210    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="config-reloader"
Sep 11 12:58:24.379263 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379226    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="thanos-sidecar"
Sep 11 12:58:24.379263 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379235    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="thanos-sidecar"
Sep 11 12:58:24.379263 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379254    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="kube-rbac-proxy-web"
Sep 11 12:58:24.379263 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379263    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="kube-rbac-proxy-web"
Sep 11 12:58:24.379618 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379279    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="kube-rbac-proxy-thanos"
Sep 11 12:58:24.379618 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379288    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="kube-rbac-proxy-thanos"
Sep 11 12:58:24.379618 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379301    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="init-config-reloader"
Sep 11 12:58:24.379618 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379311    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="init-config-reloader"
Sep 11 12:58:24.379618 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379326    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="kube-rbac-proxy"
Sep 11 12:58:24.379618 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379334    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="kube-rbac-proxy"
Sep 11 12:58:24.379618 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379413    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="config-reloader"
Sep 11 12:58:24.379618 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379447    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="prometheus"
Sep 11 12:58:24.379618 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379457    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="kube-rbac-proxy"
Sep 11 12:58:24.379618 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379469    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="kube-rbac-proxy-web"
Sep 11 12:58:24.379618 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379480    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="kube-rbac-proxy-thanos"
Sep 11 12:58:24.379618 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.379490    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" containerName="thanos-sidecar"
Sep 11 12:58:24.383144 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.383127    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.385750 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.385730    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 12:58:24.386172 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.386155    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 12:58:24.386252 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.386208    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9e4e3jdrjlev5\""
Sep 11 12:58:24.386252 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.386229    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 12:58:24.386361 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.386293    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 12:58:24.386712 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.386692    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 12:58:24.386790 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.386771    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 12:58:24.386871 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.386854    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 12:58:24.386939 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.386923    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 12:58:24.386990 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.386981    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 12:58:24.387274 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.387256    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 12:58:24.387381 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.387355    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 12:58:24.387522 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.387460    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pr8vc\""
Sep 11 12:58:24.387567 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.387550    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 12:58:24.389155 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.389139    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 12:58:24.392513 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.392496    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 12:58:24.403309 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.403281    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 12:58:24.410869 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.410848    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.410984 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.410959    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411064 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411002    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411120 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411090    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411161 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411131    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411203 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411167    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fbaa8dc3-cb17-47e6-9b02-5499e3694961-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411203 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411184    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411299 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411204    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411299 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411239    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411299 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411255    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411450 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411363    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fbaa8dc3-cb17-47e6-9b02-5499e3694961-config-out\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411450 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411430    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411553 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411463    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-config\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411553 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411510    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411643 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411563    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411643 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411599    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411643 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411620    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411772 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411686    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-web-config\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411887 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411862    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fbaa8dc3-cb17-47e6-9b02-5499e3694961-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.411934 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.411897    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54th8\" (UniqueName: \"kubernetes.io/projected/fbaa8dc3-cb17-47e6-9b02-5499e3694961-kube-api-access-54th8\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.474514 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.474487    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4" path="/var/lib/kubelet/pods/68dd05fb-9c8b-48ca-b3c9-91cda7d8ecf4/volumes"
Sep 11 12:58:24.512802 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.512780    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.512873 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.512806    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.512873 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.512823    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-web-config\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513115 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513093    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fbaa8dc3-cb17-47e6-9b02-5499e3694961-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513199 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513125    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54th8\" (UniqueName: \"kubernetes.io/projected/fbaa8dc3-cb17-47e6-9b02-5499e3694961-kube-api-access-54th8\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513274 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513210    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513274 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513252    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513369 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513369 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513315    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513369 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513342    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513574 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513407    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fbaa8dc3-cb17-47e6-9b02-5499e3694961-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513574 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513451    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513574 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513502    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513574 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513527    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513574 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513536    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fbaa8dc3-cb17-47e6-9b02-5499e3694961-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513574 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513553    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513614    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fbaa8dc3-cb17-47e6-9b02-5499e3694961-config-out\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513649    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513686    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-config\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513709    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.513857 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513735    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.514099 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.513992    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.514099 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.514005    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.514099 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.514087    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.516680 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.515193    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.516680 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.516347    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.516914 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.516886    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fbaa8dc3-cb17-47e6-9b02-5499e3694961-config-out\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.517361 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.517332    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.518242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.517461    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-web-config\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.518242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.517470    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.518242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.517535    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.518242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.517903    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.518242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.518091    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-config\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.518242 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.518200    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.518537 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.518498    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.518696 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.518675    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fbaa8dc3-cb17-47e6-9b02-5499e3694961-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.518898 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.518881    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.519501 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.519482    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fbaa8dc3-cb17-47e6-9b02-5499e3694961-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.519610 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.519594    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fbaa8dc3-cb17-47e6-9b02-5499e3694961-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.521773 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.521751    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54th8\" (UniqueName: \"kubernetes.io/projected/fbaa8dc3-cb17-47e6-9b02-5499e3694961-kube-api-access-54th8\") pod \"prometheus-k8s-0\" (UID: \"fbaa8dc3-cb17-47e6-9b02-5499e3694961\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.693324 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.693271    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:58:24.976395 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:24.976282    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 12:58:24.979053 ip-10-0-143-6 kubenswrapper[2582]: W0911 12:58:24.979026    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbaa8dc3_cb17_47e6_9b02_5499e3694961.slice/crio-69ca346c5084ec6ecdafa8e437af2399afaf6eddfcff9a04d8590f857d3c5fe3 WatchSource:0}: Error finding container 69ca346c5084ec6ecdafa8e437af2399afaf6eddfcff9a04d8590f857d3c5fe3: Status 404 returned error can't find the container with id 69ca346c5084ec6ecdafa8e437af2399afaf6eddfcff9a04d8590f857d3c5fe3
Sep 11 12:58:25.318590 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:25.318521    2582 generic.go:356] "Generic (PLEG): container finished" podID="fbaa8dc3-cb17-47e6-9b02-5499e3694961" containerID="52b9ca5cb057f5da1605d5e6c442cdf4076dbbdd24dc126582213d02cec1ef2a" exitCode=0
Sep 11 12:58:25.318981 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:25.318608    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbaa8dc3-cb17-47e6-9b02-5499e3694961","Type":"ContainerDied","Data":"52b9ca5cb057f5da1605d5e6c442cdf4076dbbdd24dc126582213d02cec1ef2a"}
Sep 11 12:58:25.318981 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:25.318685    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbaa8dc3-cb17-47e6-9b02-5499e3694961","Type":"ContainerStarted","Data":"69ca346c5084ec6ecdafa8e437af2399afaf6eddfcff9a04d8590f857d3c5fe3"}
Sep 11 12:58:26.327690 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:26.327651    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbaa8dc3-cb17-47e6-9b02-5499e3694961","Type":"ContainerStarted","Data":"b1bfd5b0bc50af910f0f5944d4a7aa3e51f2fee66ddf6969a86aafc8edc34476"}
Sep 11 12:58:26.328039 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:26.327742    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbaa8dc3-cb17-47e6-9b02-5499e3694961","Type":"ContainerStarted","Data":"ff97e3e108b76af554dd372161d86a339b654bf82de38b15c7335ed351703969"}
Sep 11 12:58:26.328039 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:26.327782    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbaa8dc3-cb17-47e6-9b02-5499e3694961","Type":"ContainerStarted","Data":"3f9284aab4637a5455af4d949772281f35b3737513f921fd08b33e58282a5eec"}
Sep 11 12:58:26.328039 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:26.327825    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbaa8dc3-cb17-47e6-9b02-5499e3694961","Type":"ContainerStarted","Data":"9d4328a503f09100a1455211a948d007f6e7eb9376a3e18ad0d4b7d0b5d1a279"}
Sep 11 12:58:26.328039 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:26.327860    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbaa8dc3-cb17-47e6-9b02-5499e3694961","Type":"ContainerStarted","Data":"6842c657a8c054fe817b76fdb474cc2bbfe1818bd5c9e5f57e6e481d0dbcfe3c"}
Sep 11 12:58:26.328039 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:26.327897    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbaa8dc3-cb17-47e6-9b02-5499e3694961","Type":"ContainerStarted","Data":"8336b01514be97c044e4d36c472de27ca05279f1d32ee60713b6be99cf51137a"}
Sep 11 12:58:26.363338 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:26.363286    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.3444917370000002 podStartE2EDuration="2.363269012s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.777275ms" isStatefulPod=false podCreationTimestamp="2026-09-11 12:58:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:58:26.360308039 +0000 UTC m=+160.300009372" watchObservedRunningTime="2026-09-11 12:58:26.363269012 +0000 UTC m=+160.302970345"
Sep 11 12:58:29.693541 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:58:29.693497    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:59:24.693962 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:59:24.693924    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:59:24.708360 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:59:24.708336    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 12:59:25.590736 ip-10-0-143-6 kubenswrapper[2582]: I0911 12:59:25.590701    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:00:46.428431 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:46.428374    2582 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 13:00:48.774120 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:48.774081    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ht9ww"]
Sep 11 13:00:48.777334 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:48.777315    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ht9ww"
Sep 11 13:00:48.785615 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:00:48.785587    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"openshift-lws-operator-fd8ccff4c-ht9ww\" is forbidden: User \"system:node:ip-10-0-143-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-143-6.ec2.internal' and this object" podUID="4ad5ab77-3c1c-444e-8007-4aed0811f02b" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ht9ww"
Sep 11 13:00:48.786236 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:48.786221    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:00:48.786644 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:48.786631    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 13:00:48.787311 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:48.787298    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-ghj9d\""
Sep 11 13:00:48.806647 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:48.806617    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ht9ww"]
Sep 11 13:00:48.899771 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:48.899727    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bjtf\" (UniqueName: \"kubernetes.io/projected/4ad5ab77-3c1c-444e-8007-4aed0811f02b-kube-api-access-7bjtf\") pod \"openshift-lws-operator-fd8ccff4c-ht9ww\" (UID: \"4ad5ab77-3c1c-444e-8007-4aed0811f02b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ht9ww"
Sep 11 13:00:48.899771 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:48.899776    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ad5ab77-3c1c-444e-8007-4aed0811f02b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ht9ww\" (UID: \"4ad5ab77-3c1c-444e-8007-4aed0811f02b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ht9ww"
Sep 11 13:00:49.001051 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:49.001008    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bjtf\" (UniqueName: \"kubernetes.io/projected/4ad5ab77-3c1c-444e-8007-4aed0811f02b-kube-api-access-7bjtf\") pod \"openshift-lws-operator-fd8ccff4c-ht9ww\" (UID: \"4ad5ab77-3c1c-444e-8007-4aed0811f02b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ht9ww"
Sep 11 13:00:49.001051 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:49.001057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ad5ab77-3c1c-444e-8007-4aed0811f02b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ht9ww\" (UID: \"4ad5ab77-3c1c-444e-8007-4aed0811f02b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ht9ww"
Sep 11 13:00:49.001497 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:49.001477    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ad5ab77-3c1c-444e-8007-4aed0811f02b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ht9ww\" (UID: \"4ad5ab77-3c1c-444e-8007-4aed0811f02b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ht9ww"
Sep 11 13:00:49.010914 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:49.010889    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bjtf\" (UniqueName: \"kubernetes.io/projected/4ad5ab77-3c1c-444e-8007-4aed0811f02b-kube-api-access-7bjtf\") pod \"openshift-lws-operator-fd8ccff4c-ht9ww\" (UID: \"4ad5ab77-3c1c-444e-8007-4aed0811f02b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ht9ww"
Sep 11 13:00:49.093658 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:49.093578    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ht9ww"
Sep 11 13:00:49.362813 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:49.362747    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ht9ww"]
Sep 11 13:00:49.365507 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:00:49.365478    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ad5ab77_3c1c_444e_8007_4aed0811f02b.slice/crio-e80261dd1a7dab5a88a0e95e6c256646e5e7ecfe93c3ec3c41ef24bf7ffa7bc9 WatchSource:0}: Error finding container e80261dd1a7dab5a88a0e95e6c256646e5e7ecfe93c3ec3c41ef24bf7ffa7bc9: Status 404 returned error can't find the container with id e80261dd1a7dab5a88a0e95e6c256646e5e7ecfe93c3ec3c41ef24bf7ffa7bc9
Sep 11 13:00:49.370468 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:49.368259    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:00:49.918897 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:49.918861    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ht9ww" event={"ID":"4ad5ab77-3c1c-444e-8007-4aed0811f02b","Type":"ContainerStarted","Data":"e80261dd1a7dab5a88a0e95e6c256646e5e7ecfe93c3ec3c41ef24bf7ffa7bc9"}
Sep 11 13:00:53.940612 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:53.940571    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ht9ww" event={"ID":"4ad5ab77-3c1c-444e-8007-4aed0811f02b","Type":"ContainerStarted","Data":"ea16c21035b1085ca65a00f4a72005e9d2451ff74184f9f398d91279f19eba06"}
Sep 11 13:00:53.964147 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:00:53.964073    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ht9ww" podStartSLOduration=2.4125641 podStartE2EDuration="5.964054467s" totalImagesPullingTime="3.551490367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:00:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:00:53.963312352 +0000 UTC m=+307.903013685" watchObservedRunningTime="2026-09-11 13:00:53.964054467 +0000 UTC m=+307.903755801"
Sep 11 13:01:13.633866 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.633826    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"]
Sep 11 13:01:13.639485 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.639467    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.645383 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.645351    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 13:01:13.645514 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.645353    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xt7p7\""
Sep 11 13:01:13.645514 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.645353    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 13:01:13.645721 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.645704    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 13:01:13.645801 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.645710    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 13:01:13.668861 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.668832    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"]
Sep 11 13:01:13.707918 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.707891    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55a86612-fe83-4731-a2df-1e5c9e79c1fb-webhook-cert\") pod \"opendatahub-operator-controller-manager-56f55dc566-2l7wq\" (UID: \"55a86612-fe83-4731-a2df-1e5c9e79c1fb\") " pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.708024 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.707942    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/55a86612-fe83-4731-a2df-1e5c9e79c1fb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56f55dc566-2l7wq\" (UID: \"55a86612-fe83-4731-a2df-1e5c9e79c1fb\") " pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.708024 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.707980    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55a86612-fe83-4731-a2df-1e5c9e79c1fb-tmp\") pod \"opendatahub-operator-controller-manager-56f55dc566-2l7wq\" (UID: \"55a86612-fe83-4731-a2df-1e5c9e79c1fb\") " pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.708024 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.708001    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/55a86612-fe83-4731-a2df-1e5c9e79c1fb-manifests\") pod \"opendatahub-operator-controller-manager-56f55dc566-2l7wq\" (UID: \"55a86612-fe83-4731-a2df-1e5c9e79c1fb\") " pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.708127 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.708029    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h62dw\" (UniqueName: \"kubernetes.io/projected/55a86612-fe83-4731-a2df-1e5c9e79c1fb-kube-api-access-h62dw\") pod \"opendatahub-operator-controller-manager-56f55dc566-2l7wq\" (UID: \"55a86612-fe83-4731-a2df-1e5c9e79c1fb\") " pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.808659 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.808627    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/55a86612-fe83-4731-a2df-1e5c9e79c1fb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56f55dc566-2l7wq\" (UID: \"55a86612-fe83-4731-a2df-1e5c9e79c1fb\") " pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.808659 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.808660    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55a86612-fe83-4731-a2df-1e5c9e79c1fb-tmp\") pod \"opendatahub-operator-controller-manager-56f55dc566-2l7wq\" (UID: \"55a86612-fe83-4731-a2df-1e5c9e79c1fb\") " pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.808861 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.808682    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/55a86612-fe83-4731-a2df-1e5c9e79c1fb-manifests\") pod \"opendatahub-operator-controller-manager-56f55dc566-2l7wq\" (UID: \"55a86612-fe83-4731-a2df-1e5c9e79c1fb\") " pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.808861 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.808802    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h62dw\" (UniqueName: \"kubernetes.io/projected/55a86612-fe83-4731-a2df-1e5c9e79c1fb-kube-api-access-h62dw\") pod \"opendatahub-operator-controller-manager-56f55dc566-2l7wq\" (UID: \"55a86612-fe83-4731-a2df-1e5c9e79c1fb\") " pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.808970 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.808891    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55a86612-fe83-4731-a2df-1e5c9e79c1fb-webhook-cert\") pod \"opendatahub-operator-controller-manager-56f55dc566-2l7wq\" (UID: \"55a86612-fe83-4731-a2df-1e5c9e79c1fb\") " pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.809056 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.809037    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/55a86612-fe83-4731-a2df-1e5c9e79c1fb-manifests\") pod \"opendatahub-operator-controller-manager-56f55dc566-2l7wq\" (UID: \"55a86612-fe83-4731-a2df-1e5c9e79c1fb\") " pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.810912 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.810877    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55a86612-fe83-4731-a2df-1e5c9e79c1fb-tmp\") pod \"opendatahub-operator-controller-manager-56f55dc566-2l7wq\" (UID: \"55a86612-fe83-4731-a2df-1e5c9e79c1fb\") " pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.811194 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.811177    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55a86612-fe83-4731-a2df-1e5c9e79c1fb-webhook-cert\") pod \"opendatahub-operator-controller-manager-56f55dc566-2l7wq\" (UID: \"55a86612-fe83-4731-a2df-1e5c9e79c1fb\") " pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.811270 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.811256    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/55a86612-fe83-4731-a2df-1e5c9e79c1fb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56f55dc566-2l7wq\" (UID: \"55a86612-fe83-4731-a2df-1e5c9e79c1fb\") " pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.818056 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.818032    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h62dw\" (UniqueName: \"kubernetes.io/projected/55a86612-fe83-4731-a2df-1e5c9e79c1fb-kube-api-access-h62dw\") pod \"opendatahub-operator-controller-manager-56f55dc566-2l7wq\" (UID: \"55a86612-fe83-4731-a2df-1e5c9e79c1fb\") " pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:13.949169 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:13.949084    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:14.233312 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:14.233288    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"]
Sep 11 13:01:14.234723 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:01:14.234696    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55a86612_fe83_4731_a2df_1e5c9e79c1fb.slice/crio-7c6a681337391940b574503ab1cc8e216f1f40fd66602460a7f03e740b665273 WatchSource:0}: Error finding container 7c6a681337391940b574503ab1cc8e216f1f40fd66602460a7f03e740b665273: Status 404 returned error can't find the container with id 7c6a681337391940b574503ab1cc8e216f1f40fd66602460a7f03e740b665273
Sep 11 13:01:15.029250 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:15.029197    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq" event={"ID":"55a86612-fe83-4731-a2df-1e5c9e79c1fb","Type":"ContainerStarted","Data":"7c6a681337391940b574503ab1cc8e216f1f40fd66602460a7f03e740b665273"}
Sep 11 13:01:18.044551 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:18.044510    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq" event={"ID":"55a86612-fe83-4731-a2df-1e5c9e79c1fb","Type":"ContainerStarted","Data":"a0bdb69589b9e0871fd86c0b09cea7ef9c0c5996a82c6ebc45014e6eb68d85fb"}
Sep 11 13:01:19.050142 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:19.050107    2582 generic.go:356] "Generic (PLEG): container finished" podID="55a86612-fe83-4731-a2df-1e5c9e79c1fb" containerID="a0bdb69589b9e0871fd86c0b09cea7ef9c0c5996a82c6ebc45014e6eb68d85fb" exitCode=0
Sep 11 13:01:19.050633 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:19.050199    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq" event={"ID":"55a86612-fe83-4731-a2df-1e5c9e79c1fb","Type":"ContainerDied","Data":"a0bdb69589b9e0871fd86c0b09cea7ef9c0c5996a82c6ebc45014e6eb68d85fb"}
Sep 11 13:01:20.056919 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:20.056830    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq" event={"ID":"55a86612-fe83-4731-a2df-1e5c9e79c1fb","Type":"ContainerStarted","Data":"5178d7d05866026df0d4c5d661f8b5aa6aa6d93ab7c87295e2ca370f29b59fae"}
Sep 11 13:01:20.057268 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:20.057008    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:20.085376 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:20.085330    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq" podStartSLOduration=2.417401586 podStartE2EDuration="7.085315025s" totalImagesPullingTime="3.66421096s" totalInitContainerRuntime="1.003702479s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:01:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:01:20.081865916 +0000 UTC m=+334.021567251" watchObservedRunningTime="2026-09-11 13:01:20.085315025 +0000 UTC m=+334.025016358"
Sep 11 13:01:31.064527 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:31.064493    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56f55dc566-2l7wq"
Sep 11 13:01:41.868805 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:41.868713    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"]
Sep 11 13:01:41.871825 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:41.871805    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:41.874800 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:41.874780    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 13:01:41.876193 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:41.876169    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-dqx96\""
Sep 11 13:01:41.876193 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:41.876178    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 13:01:41.876355 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:41.876259    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 13:01:41.884006 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:41.883986    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"]
Sep 11 13:01:41.928145 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:41.928108    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/786f3263-387a-4421-95ea-4bc7555a3b55-metrics-cert\") pod \"lws-controller-manager-5cd99cffb8-82pss\" (UID: \"786f3263-387a-4421-95ea-4bc7555a3b55\") " pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:41.928305 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:41.928164    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/786f3263-387a-4421-95ea-4bc7555a3b55-cert\") pod \"lws-controller-manager-5cd99cffb8-82pss\" (UID: \"786f3263-387a-4421-95ea-4bc7555a3b55\") " pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:41.928305 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:41.928226    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/786f3263-387a-4421-95ea-4bc7555a3b55-manager-config\") pod \"lws-controller-manager-5cd99cffb8-82pss\" (UID: \"786f3263-387a-4421-95ea-4bc7555a3b55\") " pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:41.928305 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:41.928282    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z7t5\" (UniqueName: \"kubernetes.io/projected/786f3263-387a-4421-95ea-4bc7555a3b55-kube-api-access-6z7t5\") pod \"lws-controller-manager-5cd99cffb8-82pss\" (UID: \"786f3263-387a-4421-95ea-4bc7555a3b55\") " pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:42.029155 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:42.029120    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/786f3263-387a-4421-95ea-4bc7555a3b55-metrics-cert\") pod \"lws-controller-manager-5cd99cffb8-82pss\" (UID: \"786f3263-387a-4421-95ea-4bc7555a3b55\") " pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:42.029309 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:42.029195    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/786f3263-387a-4421-95ea-4bc7555a3b55-cert\") pod \"lws-controller-manager-5cd99cffb8-82pss\" (UID: \"786f3263-387a-4421-95ea-4bc7555a3b55\") " pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:42.029309 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:42.029240    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/786f3263-387a-4421-95ea-4bc7555a3b55-manager-config\") pod \"lws-controller-manager-5cd99cffb8-82pss\" (UID: \"786f3263-387a-4421-95ea-4bc7555a3b55\") " pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:42.029309 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:42.029296    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z7t5\" (UniqueName: \"kubernetes.io/projected/786f3263-387a-4421-95ea-4bc7555a3b55-kube-api-access-6z7t5\") pod \"lws-controller-manager-5cd99cffb8-82pss\" (UID: \"786f3263-387a-4421-95ea-4bc7555a3b55\") " pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:42.029972 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:42.029948    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/786f3263-387a-4421-95ea-4bc7555a3b55-manager-config\") pod \"lws-controller-manager-5cd99cffb8-82pss\" (UID: \"786f3263-387a-4421-95ea-4bc7555a3b55\") " pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:42.031660 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:42.031640    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/786f3263-387a-4421-95ea-4bc7555a3b55-metrics-cert\") pod \"lws-controller-manager-5cd99cffb8-82pss\" (UID: \"786f3263-387a-4421-95ea-4bc7555a3b55\") " pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:42.031756 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:42.031745    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/786f3263-387a-4421-95ea-4bc7555a3b55-cert\") pod \"lws-controller-manager-5cd99cffb8-82pss\" (UID: \"786f3263-387a-4421-95ea-4bc7555a3b55\") " pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:42.038595 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:42.038576    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z7t5\" (UniqueName: \"kubernetes.io/projected/786f3263-387a-4421-95ea-4bc7555a3b55-kube-api-access-6z7t5\") pod \"lws-controller-manager-5cd99cffb8-82pss\" (UID: \"786f3263-387a-4421-95ea-4bc7555a3b55\") " pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:42.180601 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:42.180511    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:42.456085 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:42.456015    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"]
Sep 11 13:01:42.459544 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:01:42.459517    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod786f3263_387a_4421_95ea_4bc7555a3b55.slice/crio-bf9050e811e15fd7aa94ae1dddcaa5dac0c3665d461e57dfe8629b0f4191bb88 WatchSource:0}: Error finding container bf9050e811e15fd7aa94ae1dddcaa5dac0c3665d461e57dfe8629b0f4191bb88: Status 404 returned error can't find the container with id bf9050e811e15fd7aa94ae1dddcaa5dac0c3665d461e57dfe8629b0f4191bb88
Sep 11 13:01:43.159465 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:43.159414    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss" event={"ID":"786f3263-387a-4421-95ea-4bc7555a3b55","Type":"ContainerStarted","Data":"bf9050e811e15fd7aa94ae1dddcaa5dac0c3665d461e57dfe8629b0f4191bb88"}
Sep 11 13:01:45.170125 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:45.170083    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss" event={"ID":"786f3263-387a-4421-95ea-4bc7555a3b55","Type":"ContainerStarted","Data":"e75c95925927b93855dc992a41687fdd9a54cfec0770ecdf228a4d0162200aa6"}
Sep 11 13:01:45.170597 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:45.170145    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:45.203539 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:45.203490    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss" podStartSLOduration=2.474779039 podStartE2EDuration="4.203473851s" totalImagesPullingTime="1.728694812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:01:45.200481562 +0000 UTC m=+359.140182895" watchObservedRunningTime="2026-09-11 13:01:45.203473851 +0000 UTC m=+359.143175183"
Sep 11 13:01:45.529000 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:45.528966    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-v6gnm"]
Sep 11 13:01:45.532531 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:45.532504    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-v6gnm"
Sep 11 13:01:45.535284 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:45.535248    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 11 13:01:45.535405 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:45.535318    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-mj5cs\""
Sep 11 13:01:45.536027 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:45.536003    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-v6gnm"]
Sep 11 13:01:45.556455 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:45.556396    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n24rj\" (UniqueName: \"kubernetes.io/projected/d9e643ac-fe88-4914-84ca-226e2d9a2059-kube-api-access-n24rj\") pod \"odh-model-controller-7998c67dc4-v6gnm\" (UID: \"d9e643ac-fe88-4914-84ca-226e2d9a2059\") " pod="opendatahub/odh-model-controller-7998c67dc4-v6gnm"
Sep 11 13:01:45.556611 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:45.556503    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d9e643ac-fe88-4914-84ca-226e2d9a2059-cert\") pod \"odh-model-controller-7998c67dc4-v6gnm\" (UID: \"d9e643ac-fe88-4914-84ca-226e2d9a2059\") " pod="opendatahub/odh-model-controller-7998c67dc4-v6gnm"
Sep 11 13:01:45.657328 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:45.657294    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n24rj\" (UniqueName: \"kubernetes.io/projected/d9e643ac-fe88-4914-84ca-226e2d9a2059-kube-api-access-n24rj\") pod \"odh-model-controller-7998c67dc4-v6gnm\" (UID: \"d9e643ac-fe88-4914-84ca-226e2d9a2059\") " pod="opendatahub/odh-model-controller-7998c67dc4-v6gnm"
Sep 11 13:01:45.657551 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:45.657404    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d9e643ac-fe88-4914-84ca-226e2d9a2059-cert\") pod \"odh-model-controller-7998c67dc4-v6gnm\" (UID: \"d9e643ac-fe88-4914-84ca-226e2d9a2059\") " pod="opendatahub/odh-model-controller-7998c67dc4-v6gnm"
Sep 11 13:01:45.657551 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:01:45.657542    2582 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 11 13:01:45.657689 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:01:45.657623    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9e643ac-fe88-4914-84ca-226e2d9a2059-cert podName:d9e643ac-fe88-4914-84ca-226e2d9a2059 nodeName:}" failed. No retries permitted until 2026-09-11 13:01:46.157603961 +0000 UTC m=+360.097305295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d9e643ac-fe88-4914-84ca-226e2d9a2059-cert") pod "odh-model-controller-7998c67dc4-v6gnm" (UID: "d9e643ac-fe88-4914-84ca-226e2d9a2059") : secret "odh-model-controller-webhook-cert" not found
Sep 11 13:01:45.672396 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:45.672369    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n24rj\" (UniqueName: \"kubernetes.io/projected/d9e643ac-fe88-4914-84ca-226e2d9a2059-kube-api-access-n24rj\") pod \"odh-model-controller-7998c67dc4-v6gnm\" (UID: \"d9e643ac-fe88-4914-84ca-226e2d9a2059\") " pod="opendatahub/odh-model-controller-7998c67dc4-v6gnm"
Sep 11 13:01:46.059084 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.059045    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bkmhq"]
Sep 11 13:01:46.062444 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.062413    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bkmhq"
Sep 11 13:01:46.065777 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.065757    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 11 13:01:46.080462 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.080444    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bkmhq"]
Sep 11 13:01:46.160333 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.160308    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3924b26-63e9-459e-8f20-4e516beffdac-tls-certs\") pod \"model-serving-api-54d89bd5fc-bkmhq\" (UID: \"c3924b26-63e9-459e-8f20-4e516beffdac\") " pod="opendatahub/model-serving-api-54d89bd5fc-bkmhq"
Sep 11 13:01:46.160487 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.160341    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2mmt\" (UniqueName: \"kubernetes.io/projected/c3924b26-63e9-459e-8f20-4e516beffdac-kube-api-access-k2mmt\") pod \"model-serving-api-54d89bd5fc-bkmhq\" (UID: \"c3924b26-63e9-459e-8f20-4e516beffdac\") " pod="opendatahub/model-serving-api-54d89bd5fc-bkmhq"
Sep 11 13:01:46.160487 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.160453    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d9e643ac-fe88-4914-84ca-226e2d9a2059-cert\") pod \"odh-model-controller-7998c67dc4-v6gnm\" (UID: \"d9e643ac-fe88-4914-84ca-226e2d9a2059\") " pod="opendatahub/odh-model-controller-7998c67dc4-v6gnm"
Sep 11 13:01:46.162777 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.162753    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d9e643ac-fe88-4914-84ca-226e2d9a2059-cert\") pod \"odh-model-controller-7998c67dc4-v6gnm\" (UID: \"d9e643ac-fe88-4914-84ca-226e2d9a2059\") " pod="opendatahub/odh-model-controller-7998c67dc4-v6gnm"
Sep 11 13:01:46.261427 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.261401    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3924b26-63e9-459e-8f20-4e516beffdac-tls-certs\") pod \"model-serving-api-54d89bd5fc-bkmhq\" (UID: \"c3924b26-63e9-459e-8f20-4e516beffdac\") " pod="opendatahub/model-serving-api-54d89bd5fc-bkmhq"
Sep 11 13:01:46.261861 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.261463    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2mmt\" (UniqueName: \"kubernetes.io/projected/c3924b26-63e9-459e-8f20-4e516beffdac-kube-api-access-k2mmt\") pod \"model-serving-api-54d89bd5fc-bkmhq\" (UID: \"c3924b26-63e9-459e-8f20-4e516beffdac\") " pod="opendatahub/model-serving-api-54d89bd5fc-bkmhq"
Sep 11 13:01:46.263935 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.263906    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3924b26-63e9-459e-8f20-4e516beffdac-tls-certs\") pod \"model-serving-api-54d89bd5fc-bkmhq\" (UID: \"c3924b26-63e9-459e-8f20-4e516beffdac\") " pod="opendatahub/model-serving-api-54d89bd5fc-bkmhq"
Sep 11 13:01:46.271183 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.271158    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2mmt\" (UniqueName: \"kubernetes.io/projected/c3924b26-63e9-459e-8f20-4e516beffdac-kube-api-access-k2mmt\") pod \"model-serving-api-54d89bd5fc-bkmhq\" (UID: \"c3924b26-63e9-459e-8f20-4e516beffdac\") " pod="opendatahub/model-serving-api-54d89bd5fc-bkmhq"
Sep 11 13:01:46.372736 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.372658    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bkmhq"
Sep 11 13:01:46.447314 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.447256    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-mj5cs\""
Sep 11 13:01:46.455161 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.455137    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-v6gnm"
Sep 11 13:01:46.644323 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.644302    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bkmhq"]
Sep 11 13:01:46.646333 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:01:46.646304    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3924b26_63e9_459e_8f20_4e516beffdac.slice/crio-6a80da10df1a3cad708d3843af0d2b00035dace80034a0a1202e8b9da56c6d23 WatchSource:0}: Error finding container 6a80da10df1a3cad708d3843af0d2b00035dace80034a0a1202e8b9da56c6d23: Status 404 returned error can't find the container with id 6a80da10df1a3cad708d3843af0d2b00035dace80034a0a1202e8b9da56c6d23
Sep 11 13:01:46.727768 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:46.727578    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-v6gnm"]
Sep 11 13:01:46.730465 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:01:46.730436    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9e643ac_fe88_4914_84ca_226e2d9a2059.slice/crio-9012d9f49e7332b9edfe8bf6266541ba9681900f83989f1945f8d115f400397f WatchSource:0}: Error finding container 9012d9f49e7332b9edfe8bf6266541ba9681900f83989f1945f8d115f400397f: Status 404 returned error can't find the container with id 9012d9f49e7332b9edfe8bf6266541ba9681900f83989f1945f8d115f400397f
Sep 11 13:01:47.179932 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:47.179891    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-v6gnm" event={"ID":"d9e643ac-fe88-4914-84ca-226e2d9a2059","Type":"ContainerStarted","Data":"9012d9f49e7332b9edfe8bf6266541ba9681900f83989f1945f8d115f400397f"}
Sep 11 13:01:47.181116 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:47.181092    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bkmhq" event={"ID":"c3924b26-63e9-459e-8f20-4e516beffdac","Type":"ContainerStarted","Data":"6a80da10df1a3cad708d3843af0d2b00035dace80034a0a1202e8b9da56c6d23"}
Sep 11 13:01:51.204574 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:51.204534    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-v6gnm" event={"ID":"d9e643ac-fe88-4914-84ca-226e2d9a2059","Type":"ContainerStarted","Data":"8efc276368338321c7a7a17fea5c09d4b3fa2891f900fa1560afefe02b1c873f"}
Sep 11 13:01:51.205030 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:51.204648    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-v6gnm"
Sep 11 13:01:51.206089 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:51.206068    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bkmhq" event={"ID":"c3924b26-63e9-459e-8f20-4e516beffdac","Type":"ContainerStarted","Data":"7758c3147f4a5ac0a39b069a4332ef3ca3af4d60cb83d28524851ae5ee2032f5"}
Sep 11 13:01:51.206215 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:51.206200    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-bkmhq"
Sep 11 13:01:51.230348 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:51.230304    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-v6gnm" podStartSLOduration=2.466642131 podStartE2EDuration="6.230292131s" totalImagesPullingTime="3.76365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:01:51.228376482 +0000 UTC m=+365.168077812" watchObservedRunningTime="2026-09-11 13:01:51.230292131 +0000 UTC m=+365.169993461"
Sep 11 13:01:51.249021 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:51.248967    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-bkmhq" podStartSLOduration=1.405305548 podStartE2EDuration="5.248954345s" totalImagesPullingTime="3.843648797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:01:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:01:51.248848146 +0000 UTC m=+365.188549481" watchObservedRunningTime="2026-09-11 13:01:51.248954345 +0000 UTC m=+365.188655678"
Sep 11 13:01:55.735816 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.735782    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4"]
Sep 11 13:01:55.738400 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.738382    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4"
Sep 11 13:01:55.741285 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.741256    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 11 13:01:55.741387 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.741258    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-tmd6w\""
Sep 11 13:01:55.752776 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.752755    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt"]
Sep 11 13:01:55.755297 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.755276    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt"
Sep 11 13:01:55.758259 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.758238    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-msh2q\""
Sep 11 13:01:55.758930 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.758887    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 11 13:01:55.759889 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.759865    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4"]
Sep 11 13:01:55.784367 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.784343    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt"]
Sep 11 13:01:55.847360 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.847329    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nntmn\" (UniqueName: \"kubernetes.io/projected/c584055b-afaa-4aae-8c85-821b41963def-kube-api-access-nntmn\") pod \"kserve-controller-manager-d6f4b65b8-h2mb4\" (UID: \"c584055b-afaa-4aae-8c85-821b41963def\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4"
Sep 11 13:01:55.847510 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.847371    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fe1552b-1a45-4a6c-83a6-145ed7f3650f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-z2mzt\" (UID: \"5fe1552b-1a45-4a6c-83a6-145ed7f3650f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt"
Sep 11 13:01:55.847510 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.847408    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6djn\" (UniqueName: \"kubernetes.io/projected/5fe1552b-1a45-4a6c-83a6-145ed7f3650f-kube-api-access-z6djn\") pod \"llmisvc-controller-manager-777c6bd7d5-z2mzt\" (UID: \"5fe1552b-1a45-4a6c-83a6-145ed7f3650f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt"
Sep 11 13:01:55.847600 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.847527    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c584055b-afaa-4aae-8c85-821b41963def-cert\") pod \"kserve-controller-manager-d6f4b65b8-h2mb4\" (UID: \"c584055b-afaa-4aae-8c85-821b41963def\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4"
Sep 11 13:01:55.948885 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.948853    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nntmn\" (UniqueName: \"kubernetes.io/projected/c584055b-afaa-4aae-8c85-821b41963def-kube-api-access-nntmn\") pod \"kserve-controller-manager-d6f4b65b8-h2mb4\" (UID: \"c584055b-afaa-4aae-8c85-821b41963def\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4"
Sep 11 13:01:55.949001 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.948893    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fe1552b-1a45-4a6c-83a6-145ed7f3650f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-z2mzt\" (UID: \"5fe1552b-1a45-4a6c-83a6-145ed7f3650f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt"
Sep 11 13:01:55.949100 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.949080    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6djn\" (UniqueName: \"kubernetes.io/projected/5fe1552b-1a45-4a6c-83a6-145ed7f3650f-kube-api-access-z6djn\") pod \"llmisvc-controller-manager-777c6bd7d5-z2mzt\" (UID: \"5fe1552b-1a45-4a6c-83a6-145ed7f3650f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt"
Sep 11 13:01:55.949197 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.949179    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c584055b-afaa-4aae-8c85-821b41963def-cert\") pod \"kserve-controller-manager-d6f4b65b8-h2mb4\" (UID: \"c584055b-afaa-4aae-8c85-821b41963def\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4"
Sep 11 13:01:55.949330 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:01:55.949312    2582 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 11 13:01:55.949392 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:01:55.949381    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c584055b-afaa-4aae-8c85-821b41963def-cert podName:c584055b-afaa-4aae-8c85-821b41963def nodeName:}" failed. No retries permitted until 2026-09-11 13:01:56.449367484 +0000 UTC m=+370.389068799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c584055b-afaa-4aae-8c85-821b41963def-cert") pod "kserve-controller-manager-d6f4b65b8-h2mb4" (UID: "c584055b-afaa-4aae-8c85-821b41963def") : secret "kserve-webhook-server-cert" not found
Sep 11 13:01:55.951442 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.951407    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5fe1552b-1a45-4a6c-83a6-145ed7f3650f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-z2mzt\" (UID: \"5fe1552b-1a45-4a6c-83a6-145ed7f3650f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt"
Sep 11 13:01:55.963202 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.963180    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6djn\" (UniqueName: \"kubernetes.io/projected/5fe1552b-1a45-4a6c-83a6-145ed7f3650f-kube-api-access-z6djn\") pod \"llmisvc-controller-manager-777c6bd7d5-z2mzt\" (UID: \"5fe1552b-1a45-4a6c-83a6-145ed7f3650f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt"
Sep 11 13:01:55.963202 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:55.963191    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nntmn\" (UniqueName: \"kubernetes.io/projected/c584055b-afaa-4aae-8c85-821b41963def-kube-api-access-nntmn\") pod \"kserve-controller-manager-d6f4b65b8-h2mb4\" (UID: \"c584055b-afaa-4aae-8c85-821b41963def\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4"
Sep 11 13:01:56.067177 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:56.067125    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt"
Sep 11 13:01:56.176223 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:56.176197    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5cd99cffb8-82pss"
Sep 11 13:01:56.346270 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:56.346241    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt"]
Sep 11 13:01:56.347516 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:01:56.347495    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5fe1552b_1a45_4a6c_83a6_145ed7f3650f.slice/crio-25291e5139f46c020af3284184951bb66965f256fc79f43570947533afdcf82c WatchSource:0}: Error finding container 25291e5139f46c020af3284184951bb66965f256fc79f43570947533afdcf82c: Status 404 returned error can't find the container with id 25291e5139f46c020af3284184951bb66965f256fc79f43570947533afdcf82c
Sep 11 13:01:56.453434 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:56.453393    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c584055b-afaa-4aae-8c85-821b41963def-cert\") pod \"kserve-controller-manager-d6f4b65b8-h2mb4\" (UID: \"c584055b-afaa-4aae-8c85-821b41963def\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4"
Sep 11 13:01:56.455692 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:56.455673    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c584055b-afaa-4aae-8c85-821b41963def-cert\") pod \"kserve-controller-manager-d6f4b65b8-h2mb4\" (UID: \"c584055b-afaa-4aae-8c85-821b41963def\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4"
Sep 11 13:01:56.649535 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:56.649467    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4"
Sep 11 13:01:56.926671 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:56.926649    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4"]
Sep 11 13:01:56.929098 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:01:56.929072    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc584055b_afaa_4aae_8c85_821b41963def.slice/crio-a4043ac042c20635a20eb8c2eff946db50131b52457ae5a26b09fc54c9cbbf8b WatchSource:0}: Error finding container a4043ac042c20635a20eb8c2eff946db50131b52457ae5a26b09fc54c9cbbf8b: Status 404 returned error can't find the container with id a4043ac042c20635a20eb8c2eff946db50131b52457ae5a26b09fc54c9cbbf8b
Sep 11 13:01:57.245616 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:57.245572    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt" event={"ID":"5fe1552b-1a45-4a6c-83a6-145ed7f3650f","Type":"ContainerStarted","Data":"25291e5139f46c020af3284184951bb66965f256fc79f43570947533afdcf82c"}
Sep 11 13:01:57.246707 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:01:57.246680    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4" event={"ID":"c584055b-afaa-4aae-8c85-821b41963def","Type":"ContainerStarted","Data":"a4043ac042c20635a20eb8c2eff946db50131b52457ae5a26b09fc54c9cbbf8b"}
Sep 11 13:02:00.264640 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:02:00.264595    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt" event={"ID":"5fe1552b-1a45-4a6c-83a6-145ed7f3650f","Type":"ContainerStarted","Data":"b5b42d3c831afb3b753f160a6288726a6359584484a0213e37fba02746679776"}
Sep 11 13:02:00.265071 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:02:00.264743    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt"
Sep 11 13:02:00.266162 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:02:00.266139    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4" event={"ID":"c584055b-afaa-4aae-8c85-821b41963def","Type":"ContainerStarted","Data":"7bff5fd053078cccece2d939e14d7dafdc8caffd8fd6b9472de56568ff186349"}
Sep 11 13:02:00.266284 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:02:00.266250    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4"
Sep 11 13:02:00.304326 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:02:00.304286    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt" podStartSLOduration=1.811897749 podStartE2EDuration="5.304272607s" totalImagesPullingTime="3.492374858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:01:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:02:00.304070505 +0000 UTC m=+374.243771837" watchObservedRunningTime="2026-09-11 13:02:00.304272607 +0000 UTC m=+374.243973940"
Sep 11 13:02:00.337405 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:02:00.337351    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4" podStartSLOduration=2.424082021 podStartE2EDuration="5.337338111s" totalImagesPullingTime="2.91325609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:01:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:02:00.336730798 +0000 UTC m=+374.276432132" watchObservedRunningTime="2026-09-11 13:02:00.337338111 +0000 UTC m=+374.277039443"
Sep 11 13:02:02.213936 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:02:02.213905    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-v6gnm"
Sep 11 13:02:02.216435 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:02:02.216402    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-bkmhq"
Sep 11 13:02:31.275408 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:02:31.275374    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-z2mzt"
Sep 11 13:02:31.278751 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:02:31.278729    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2mb4"
Sep 11 13:03:08.667293 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:08.667255    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"]
Sep 11 13:03:08.670668 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:08.670646    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:08.674094 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:08.674068    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tt5b6\""
Sep 11 13:03:08.674519 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:08.674502    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 13:03:08.674584 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:08.674521    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 13:03:08.698027 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:08.697996    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"]
Sep 11 13:03:08.753709 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:08.753680    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp725\" (UniqueName: \"kubernetes.io/projected/9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a-kube-api-access-sp725\") pod \"kuadrant-operator-controller-manager-76f8f686bf-767r8\" (UID: \"9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:08.753805 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:08.753737    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-767r8\" (UID: \"9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:08.854812 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:08.854788    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-767r8\" (UID: \"9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:08.854888 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:08.854843    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp725\" (UniqueName: \"kubernetes.io/projected/9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a-kube-api-access-sp725\") pod \"kuadrant-operator-controller-manager-76f8f686bf-767r8\" (UID: \"9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:08.855129 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:08.855112    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-767r8\" (UID: \"9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:08.876989 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:08.876966    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp725\" (UniqueName: \"kubernetes.io/projected/9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a-kube-api-access-sp725\") pod \"kuadrant-operator-controller-manager-76f8f686bf-767r8\" (UID: \"9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:08.981074 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:08.981046    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:09.307187 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:03:09.307159    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f3e7369_bc9e_45f7_9dc7_d1ea7ae5927a.slice/crio-0238438b6eea971df518add7ddea6420aa33529f57d93fba9e9141d7ddf5c982 WatchSource:0}: Error finding container 0238438b6eea971df518add7ddea6420aa33529f57d93fba9e9141d7ddf5c982: Status 404 returned error can't find the container with id 0238438b6eea971df518add7ddea6420aa33529f57d93fba9e9141d7ddf5c982
Sep 11 13:03:09.309275 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:09.309248    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"]
Sep 11 13:03:09.602918 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:09.602836    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8" event={"ID":"9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a","Type":"ContainerStarted","Data":"0238438b6eea971df518add7ddea6420aa33529f57d93fba9e9141d7ddf5c982"}
Sep 11 13:03:13.627790 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:13.627761    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8" event={"ID":"9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a","Type":"ContainerStarted","Data":"3d3ec99db824aab75bc1822d4e183b403252d58bc06671b5124e6e20575ca601"}
Sep 11 13:03:13.628116 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:13.627826    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:13.691082 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:13.691001    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8" podStartSLOduration=1.615527103 podStartE2EDuration="5.690986251s" totalImagesPullingTime="4.075459148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:03:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:03:13.68969403 +0000 UTC m=+447.629395357" watchObservedRunningTime="2026-09-11 13:03:13.690986251 +0000 UTC m=+447.630687581"
Sep 11 13:03:24.635111 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:24.635079    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:27.097214 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.097174    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"]
Sep 11 13:03:27.097713 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.097395    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8" podUID="9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a" containerName="manager" containerID="cri-o://3d3ec99db824aab75bc1822d4e183b403252d58bc06671b5124e6e20575ca601" gracePeriod=2
Sep 11 13:03:27.111355 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.111328    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"]
Sep 11 13:03:27.126510 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:03:27.126453    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-767r8\": pods \"kuadrant-operator-controller-manager-76f8f686bf-767r8\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:27.129692 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.129667    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"]
Sep 11 13:03:27.130246 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.130228    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a" containerName="manager"
Sep 11 13:03:27.130289 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.130252    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a" containerName="manager"
Sep 11 13:03:27.130364 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.130352    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a" containerName="manager"
Sep 11 13:03:27.133729 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.133708    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"
Sep 11 13:03:27.136167 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:03:27.136141    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-767r8\" is forbidden: User \"system:node:ip-10-0-143-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-6.ec2.internal' and this object" podUID="9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:27.150737 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.150717    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"]
Sep 11 13:03:27.303962 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.303926    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9thz\" (UniqueName: \"kubernetes.io/projected/e758cd90-b26c-4657-bb70-66e8066e1061-kube-api-access-f9thz\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nqbnx\" (UID: \"e758cd90-b26c-4657-bb70-66e8066e1061\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"
Sep 11 13:03:27.304095 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.304039    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e758cd90-b26c-4657-bb70-66e8066e1061-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nqbnx\" (UID: \"e758cd90-b26c-4657-bb70-66e8066e1061\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"
Sep 11 13:03:27.335736 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.335714    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:27.340497 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:03:27.340474    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-767r8\" is forbidden: User \"system:node:ip-10-0-143-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-6.ec2.internal' and this object" podUID="9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:27.405127 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.405074    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9thz\" (UniqueName: \"kubernetes.io/projected/e758cd90-b26c-4657-bb70-66e8066e1061-kube-api-access-f9thz\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nqbnx\" (UID: \"e758cd90-b26c-4657-bb70-66e8066e1061\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"
Sep 11 13:03:27.405216 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.405135    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e758cd90-b26c-4657-bb70-66e8066e1061-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nqbnx\" (UID: \"e758cd90-b26c-4657-bb70-66e8066e1061\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"
Sep 11 13:03:27.405511 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.405493    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e758cd90-b26c-4657-bb70-66e8066e1061-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nqbnx\" (UID: \"e758cd90-b26c-4657-bb70-66e8066e1061\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"
Sep 11 13:03:27.422311 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.422284    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9thz\" (UniqueName: \"kubernetes.io/projected/e758cd90-b26c-4657-bb70-66e8066e1061-kube-api-access-f9thz\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nqbnx\" (UID: \"e758cd90-b26c-4657-bb70-66e8066e1061\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"
Sep 11 13:03:27.482063 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.482041    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"
Sep 11 13:03:27.505911 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.505892    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a-extensions-socket-volume\") pod \"9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a\" (UID: \"9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a\") "
Sep 11 13:03:27.505993 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.505954    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sp725\" (UniqueName: \"kubernetes.io/projected/9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a-kube-api-access-sp725\") pod \"9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a\" (UID: \"9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a\") "
Sep 11 13:03:27.506365 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.506346    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a" (UID: "9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:03:27.507901 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.507882    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a-kube-api-access-sp725" (OuterVolumeSpecName: "kube-api-access-sp725") pod "9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a" (UID: "9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a"). InnerVolumeSpecName "kube-api-access-sp725". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:03:27.607247 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.607218    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a-extensions-socket-volume\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:03:27.607247 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.607243    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sp725\" (UniqueName: \"kubernetes.io/projected/9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a-kube-api-access-sp725\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:03:27.698792 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.698735    2582 generic.go:356] "Generic (PLEG): container finished" podID="9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a" containerID="3d3ec99db824aab75bc1822d4e183b403252d58bc06671b5124e6e20575ca601" exitCode=0
Sep 11 13:03:27.698792 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.698783    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:27.698931 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.698811    2582 scope.go:117] "RemoveContainer" containerID="3d3ec99db824aab75bc1822d4e183b403252d58bc06671b5124e6e20575ca601"
Sep 11 13:03:27.703543 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:03:27.703515    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-767r8\" is forbidden: User \"system:node:ip-10-0-143-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-6.ec2.internal' and this object" podUID="9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:27.708396 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.708374    2582 scope.go:117] "RemoveContainer" containerID="3d3ec99db824aab75bc1822d4e183b403252d58bc06671b5124e6e20575ca601"
Sep 11 13:03:27.708694 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:03:27.708674    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d3ec99db824aab75bc1822d4e183b403252d58bc06671b5124e6e20575ca601\": container with ID starting with 3d3ec99db824aab75bc1822d4e183b403252d58bc06671b5124e6e20575ca601 not found: ID does not exist" containerID="3d3ec99db824aab75bc1822d4e183b403252d58bc06671b5124e6e20575ca601"
Sep 11 13:03:27.708777 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.708701    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d3ec99db824aab75bc1822d4e183b403252d58bc06671b5124e6e20575ca601"} err="failed to get container status \"3d3ec99db824aab75bc1822d4e183b403252d58bc06671b5124e6e20575ca601\": rpc error: code = NotFound desc = could not find container \"3d3ec99db824aab75bc1822d4e183b403252d58bc06671b5124e6e20575ca601\": container with ID starting with 3d3ec99db824aab75bc1822d4e183b403252d58bc06671b5124e6e20575ca601 not found: ID does not exist"
Sep 11 13:03:27.710658 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:03:27.710633    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-767r8\" is forbidden: User \"system:node:ip-10-0-143-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-6.ec2.internal' and this object" podUID="9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-767r8"
Sep 11 13:03:27.767411 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:27.767293    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"]
Sep 11 13:03:27.769402 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:03:27.769380    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode758cd90_b26c_4657_bb70_66e8066e1061.slice/crio-5efd6eca86321aa98d4154d9fcfdc671fd51578c917f0f119f2267b73adfb0b5 WatchSource:0}: Error finding container 5efd6eca86321aa98d4154d9fcfdc671fd51578c917f0f119f2267b73adfb0b5: Status 404 returned error can't find the container with id 5efd6eca86321aa98d4154d9fcfdc671fd51578c917f0f119f2267b73adfb0b5
Sep 11 13:03:28.482200 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:28.482167    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a" path="/var/lib/kubelet/pods/9f3e7369-bc9e-45f7-9dc7-d1ea7ae5927a/volumes"
Sep 11 13:03:28.628356 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:28.628322    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"]
Sep 11 13:03:28.631936 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:28.631915    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"
Sep 11 13:03:28.654184 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:28.654157    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"]
Sep 11 13:03:28.706053 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:28.706024    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx" event={"ID":"e758cd90-b26c-4657-bb70-66e8066e1061","Type":"ContainerStarted","Data":"9d2c7c06482e6ea7dff955b2807dcda78048724cfea363887b6567de3eb030d5"}
Sep 11 13:03:28.706171 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:28.706064    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx" event={"ID":"e758cd90-b26c-4657-bb70-66e8066e1061","Type":"ContainerStarted","Data":"5efd6eca86321aa98d4154d9fcfdc671fd51578c917f0f119f2267b73adfb0b5"}
Sep 11 13:03:28.706238 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:28.706166    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"
Sep 11 13:03:28.748135 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:28.748063    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx" podStartSLOduration=1.748050218 podStartE2EDuration="1.748050218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:03:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:03:28.746355276 +0000 UTC m=+462.686056614" watchObservedRunningTime="2026-09-11 13:03:28.748050218 +0000 UTC m=+462.687751550"
Sep 11 13:03:28.817381 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:28.817354    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3163422b-8384-4b3f-97bd-830f0ce1af77-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wfnc7\" (UID: \"3163422b-8384-4b3f-97bd-830f0ce1af77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"
Sep 11 13:03:28.817511 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:28.817405    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d986\" (UniqueName: \"kubernetes.io/projected/3163422b-8384-4b3f-97bd-830f0ce1af77-kube-api-access-4d986\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wfnc7\" (UID: \"3163422b-8384-4b3f-97bd-830f0ce1af77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"
Sep 11 13:03:28.917936 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:28.917907    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d986\" (UniqueName: \"kubernetes.io/projected/3163422b-8384-4b3f-97bd-830f0ce1af77-kube-api-access-4d986\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wfnc7\" (UID: \"3163422b-8384-4b3f-97bd-830f0ce1af77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"
Sep 11 13:03:28.918064 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:28.918034    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3163422b-8384-4b3f-97bd-830f0ce1af77-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wfnc7\" (UID: \"3163422b-8384-4b3f-97bd-830f0ce1af77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"
Sep 11 13:03:28.918386 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:28.918368    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3163422b-8384-4b3f-97bd-830f0ce1af77-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wfnc7\" (UID: \"3163422b-8384-4b3f-97bd-830f0ce1af77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"
Sep 11 13:03:28.926864 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:28.926842    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d986\" (UniqueName: \"kubernetes.io/projected/3163422b-8384-4b3f-97bd-830f0ce1af77-kube-api-access-4d986\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wfnc7\" (UID: \"3163422b-8384-4b3f-97bd-830f0ce1af77\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"
Sep 11 13:03:28.941588 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:28.941566    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"
Sep 11 13:03:29.226851 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:29.226823    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"]
Sep 11 13:03:29.228968 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:03:29.228939    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3163422b_8384_4b3f_97bd_830f0ce1af77.slice/crio-b08e07da0ab4b887713aa2dcfbf482869ec992c4c5a70c4930c09a1375baab4d WatchSource:0}: Error finding container b08e07da0ab4b887713aa2dcfbf482869ec992c4c5a70c4930c09a1375baab4d: Status 404 returned error can't find the container with id b08e07da0ab4b887713aa2dcfbf482869ec992c4c5a70c4930c09a1375baab4d
Sep 11 13:03:29.713221 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:29.713134    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7" event={"ID":"3163422b-8384-4b3f-97bd-830f0ce1af77","Type":"ContainerStarted","Data":"c92fe3f6818047819c510d2417f28feeb28de6cb0dcc53ef5d09605bd09f7f73"}
Sep 11 13:03:29.713221 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:29.713190    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7" event={"ID":"3163422b-8384-4b3f-97bd-830f0ce1af77","Type":"ContainerStarted","Data":"b08e07da0ab4b887713aa2dcfbf482869ec992c4c5a70c4930c09a1375baab4d"}
Sep 11 13:03:29.713622 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:29.713468    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"
Sep 11 13:03:29.749375 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:29.749329    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7" podStartSLOduration=1.749314285 podStartE2EDuration="1.749314285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:03:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:03:29.748312842 +0000 UTC m=+463.688014172" watchObservedRunningTime="2026-09-11 13:03:29.749314285 +0000 UTC m=+463.689015613"
Sep 11 13:03:39.714754 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:39.714718    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"
Sep 11 13:03:41.724390 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:41.724357    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"
Sep 11 13:03:41.786369 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:41.786341    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"]
Sep 11 13:03:41.786568 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:41.786544    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx" podUID="e758cd90-b26c-4657-bb70-66e8066e1061" containerName="manager" containerID="cri-o://9d2c7c06482e6ea7dff955b2807dcda78048724cfea363887b6567de3eb030d5" gracePeriod=10
Sep 11 13:03:42.019034 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.019012    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"
Sep 11 13:03:42.028974 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.028956    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9thz\" (UniqueName: \"kubernetes.io/projected/e758cd90-b26c-4657-bb70-66e8066e1061-kube-api-access-f9thz\") pod \"e758cd90-b26c-4657-bb70-66e8066e1061\" (UID: \"e758cd90-b26c-4657-bb70-66e8066e1061\") "
Sep 11 13:03:42.029043 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.029015    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e758cd90-b26c-4657-bb70-66e8066e1061-extensions-socket-volume\") pod \"e758cd90-b26c-4657-bb70-66e8066e1061\" (UID: \"e758cd90-b26c-4657-bb70-66e8066e1061\") "
Sep 11 13:03:42.029401 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.029380    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e758cd90-b26c-4657-bb70-66e8066e1061-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e758cd90-b26c-4657-bb70-66e8066e1061" (UID: "e758cd90-b26c-4657-bb70-66e8066e1061"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:03:42.030911 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.030886    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e758cd90-b26c-4657-bb70-66e8066e1061-kube-api-access-f9thz" (OuterVolumeSpecName: "kube-api-access-f9thz") pod "e758cd90-b26c-4657-bb70-66e8066e1061" (UID: "e758cd90-b26c-4657-bb70-66e8066e1061"). InnerVolumeSpecName "kube-api-access-f9thz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:03:42.130449 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.130405    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9thz\" (UniqueName: \"kubernetes.io/projected/e758cd90-b26c-4657-bb70-66e8066e1061-kube-api-access-f9thz\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:03:42.130449 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.130443    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e758cd90-b26c-4657-bb70-66e8066e1061-extensions-socket-volume\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:03:42.784771 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.784692    2582 generic.go:356] "Generic (PLEG): container finished" podID="e758cd90-b26c-4657-bb70-66e8066e1061" containerID="9d2c7c06482e6ea7dff955b2807dcda78048724cfea363887b6567de3eb030d5" exitCode=0
Sep 11 13:03:42.784771 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.784736    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"
Sep 11 13:03:42.784771 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.784750    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx" event={"ID":"e758cd90-b26c-4657-bb70-66e8066e1061","Type":"ContainerDied","Data":"9d2c7c06482e6ea7dff955b2807dcda78048724cfea363887b6567de3eb030d5"}
Sep 11 13:03:42.785209 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.784800    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx" event={"ID":"e758cd90-b26c-4657-bb70-66e8066e1061","Type":"ContainerDied","Data":"5efd6eca86321aa98d4154d9fcfdc671fd51578c917f0f119f2267b73adfb0b5"}
Sep 11 13:03:42.785209 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.784829    2582 scope.go:117] "RemoveContainer" containerID="9d2c7c06482e6ea7dff955b2807dcda78048724cfea363887b6567de3eb030d5"
Sep 11 13:03:42.793487 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.793291    2582 scope.go:117] "RemoveContainer" containerID="9d2c7c06482e6ea7dff955b2807dcda78048724cfea363887b6567de3eb030d5"
Sep 11 13:03:42.793577 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:03:42.793559    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d2c7c06482e6ea7dff955b2807dcda78048724cfea363887b6567de3eb030d5\": container with ID starting with 9d2c7c06482e6ea7dff955b2807dcda78048724cfea363887b6567de3eb030d5 not found: ID does not exist" containerID="9d2c7c06482e6ea7dff955b2807dcda78048724cfea363887b6567de3eb030d5"
Sep 11 13:03:42.793628 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.793585    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d2c7c06482e6ea7dff955b2807dcda78048724cfea363887b6567de3eb030d5"} err="failed to get container status \"9d2c7c06482e6ea7dff955b2807dcda78048724cfea363887b6567de3eb030d5\": rpc error: code = NotFound desc = could not find container \"9d2c7c06482e6ea7dff955b2807dcda78048724cfea363887b6567de3eb030d5\": container with ID starting with 9d2c7c06482e6ea7dff955b2807dcda78048724cfea363887b6567de3eb030d5 not found: ID does not exist"
Sep 11 13:03:42.810001 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.809974    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"]
Sep 11 13:03:42.825138 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:42.825116    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nqbnx"]
Sep 11 13:03:44.475644 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:03:44.475611    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e758cd90-b26c-4657-bb70-66e8066e1061" path="/var/lib/kubelet/pods/e758cd90-b26c-4657-bb70-66e8066e1061/volumes"
Sep 11 13:04:01.145622 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.145583    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-9tmxz"]
Sep 11 13:04:01.145982 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.145970    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e758cd90-b26c-4657-bb70-66e8066e1061" containerName="manager"
Sep 11 13:04:01.146023 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.145985    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e758cd90-b26c-4657-bb70-66e8066e1061" containerName="manager"
Sep 11 13:04:01.146066 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.146056    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e758cd90-b26c-4657-bb70-66e8066e1061" containerName="manager"
Sep 11 13:04:01.150327 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.150302    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-9tmxz"
Sep 11 13:04:01.153160 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.153132    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 13:04:01.153587 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.153570    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rxs7x\""
Sep 11 13:04:01.161530 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.161507    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-9tmxz"]
Sep 11 13:04:01.239050 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.239021    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-9tmxz"]
Sep 11 13:04:01.298387 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.298356    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88w5f\" (UniqueName: \"kubernetes.io/projected/baae8f20-d871-45f4-ac7f-ce1524602050-kube-api-access-88w5f\") pod \"limitador-limitador-8588749977-9tmxz\" (UID: \"baae8f20-d871-45f4-ac7f-ce1524602050\") " pod="kuadrant-system/limitador-limitador-8588749977-9tmxz"
Sep 11 13:04:01.298559 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.298401    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/baae8f20-d871-45f4-ac7f-ce1524602050-config-file\") pod \"limitador-limitador-8588749977-9tmxz\" (UID: \"baae8f20-d871-45f4-ac7f-ce1524602050\") " pod="kuadrant-system/limitador-limitador-8588749977-9tmxz"
Sep 11 13:04:01.399750 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.399685    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/baae8f20-d871-45f4-ac7f-ce1524602050-config-file\") pod \"limitador-limitador-8588749977-9tmxz\" (UID: \"baae8f20-d871-45f4-ac7f-ce1524602050\") " pod="kuadrant-system/limitador-limitador-8588749977-9tmxz"
Sep 11 13:04:01.399857 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.399758    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88w5f\" (UniqueName: \"kubernetes.io/projected/baae8f20-d871-45f4-ac7f-ce1524602050-kube-api-access-88w5f\") pod \"limitador-limitador-8588749977-9tmxz\" (UID: \"baae8f20-d871-45f4-ac7f-ce1524602050\") " pod="kuadrant-system/limitador-limitador-8588749977-9tmxz"
Sep 11 13:04:01.400289 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.400271    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/baae8f20-d871-45f4-ac7f-ce1524602050-config-file\") pod \"limitador-limitador-8588749977-9tmxz\" (UID: \"baae8f20-d871-45f4-ac7f-ce1524602050\") " pod="kuadrant-system/limitador-limitador-8588749977-9tmxz"
Sep 11 13:04:01.409547 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.409520    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88w5f\" (UniqueName: \"kubernetes.io/projected/baae8f20-d871-45f4-ac7f-ce1524602050-kube-api-access-88w5f\") pod \"limitador-limitador-8588749977-9tmxz\" (UID: \"baae8f20-d871-45f4-ac7f-ce1524602050\") " pod="kuadrant-system/limitador-limitador-8588749977-9tmxz"
Sep 11 13:04:01.460473 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.460447    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-9tmxz"
Sep 11 13:04:01.730089 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.730069    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-9tmxz"]
Sep 11 13:04:01.732521 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:04:01.732492    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaae8f20_d871_45f4_ac7f_ce1524602050.slice/crio-c0af59cfd621cf0b6e74c68cc27300dfa6e350b0e3ea9400026367aa68aa7842 WatchSource:0}: Error finding container c0af59cfd621cf0b6e74c68cc27300dfa6e350b0e3ea9400026367aa68aa7842: Status 404 returned error can't find the container with id c0af59cfd621cf0b6e74c68cc27300dfa6e350b0e3ea9400026367aa68aa7842
Sep 11 13:04:01.873152 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:01.873120    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-9tmxz" event={"ID":"baae8f20-d871-45f4-ac7f-ce1524602050","Type":"ContainerStarted","Data":"c0af59cfd621cf0b6e74c68cc27300dfa6e350b0e3ea9400026367aa68aa7842"}
Sep 11 13:04:02.101262 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:02.101201    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-pn7rp"]
Sep 11 13:04:02.104077 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:02.104058    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-pn7rp"
Sep 11 13:04:02.107488 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:02.107466    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c8tfd\""
Sep 11 13:04:02.112469 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:02.112445    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-pn7rp"]
Sep 11 13:04:02.207125 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:02.207100    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znkj7\" (UniqueName: \"kubernetes.io/projected/183e9f9a-f96c-4601-8384-c0a77d5074fe-kube-api-access-znkj7\") pod \"authorino-59c974bd48-pn7rp\" (UID: \"183e9f9a-f96c-4601-8384-c0a77d5074fe\") " pod="kuadrant-system/authorino-59c974bd48-pn7rp"
Sep 11 13:04:02.307682 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:02.307661    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znkj7\" (UniqueName: \"kubernetes.io/projected/183e9f9a-f96c-4601-8384-c0a77d5074fe-kube-api-access-znkj7\") pod \"authorino-59c974bd48-pn7rp\" (UID: \"183e9f9a-f96c-4601-8384-c0a77d5074fe\") " pod="kuadrant-system/authorino-59c974bd48-pn7rp"
Sep 11 13:04:02.317998 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:02.317979    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znkj7\" (UniqueName: \"kubernetes.io/projected/183e9f9a-f96c-4601-8384-c0a77d5074fe-kube-api-access-znkj7\") pod \"authorino-59c974bd48-pn7rp\" (UID: \"183e9f9a-f96c-4601-8384-c0a77d5074fe\") " pod="kuadrant-system/authorino-59c974bd48-pn7rp"
Sep 11 13:04:02.413997 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:02.413950    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-pn7rp"
Sep 11 13:04:02.685022 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:02.684999    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-pn7rp"]
Sep 11 13:04:02.686898 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:04:02.686873    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod183e9f9a_f96c_4601_8384_c0a77d5074fe.slice/crio-445f415b76ce99683ea2f20a5cb755b040d55e7e6ae06e1249a3dbd5ffc2260e WatchSource:0}: Error finding container 445f415b76ce99683ea2f20a5cb755b040d55e7e6ae06e1249a3dbd5ffc2260e: Status 404 returned error can't find the container with id 445f415b76ce99683ea2f20a5cb755b040d55e7e6ae06e1249a3dbd5ffc2260e
Sep 11 13:04:02.878938 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:02.878909    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-pn7rp" event={"ID":"183e9f9a-f96c-4601-8384-c0a77d5074fe","Type":"ContainerStarted","Data":"445f415b76ce99683ea2f20a5cb755b040d55e7e6ae06e1249a3dbd5ffc2260e"}
Sep 11 13:04:04.896052 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:04.895976    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-9tmxz" event={"ID":"baae8f20-d871-45f4-ac7f-ce1524602050","Type":"ContainerStarted","Data":"ac7f40134f170e7b17a0c3461cda6f0a972b7bbe17c1cdb4f61436dd0af57a08"}
Sep 11 13:04:04.896446 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:04.896095    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-9tmxz"
Sep 11 13:04:04.897681 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:04.897641    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-pn7rp" event={"ID":"183e9f9a-f96c-4601-8384-c0a77d5074fe","Type":"ContainerStarted","Data":"1b2b5fc9eb7bd68c396b3a9710c29c069f30ec47998d288088dbbdb0c2e41861"}
Sep 11 13:04:04.925912 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:04.925872    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-9tmxz" podStartSLOduration=1.132262014 podStartE2EDuration="3.92586093s" totalImagesPullingTime="2.793598916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:04.924377294 +0000 UTC m=+498.864078627" watchObservedRunningTime="2026-09-11 13:04:04.92586093 +0000 UTC m=+498.865562266"
Sep 11 13:04:04.964496 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:04.964462    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-pn7rp" podStartSLOduration=1.12336126 podStartE2EDuration="2.964450306s" totalImagesPullingTime="1.841089046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:04.963998331 +0000 UTC m=+498.903699663" watchObservedRunningTime="2026-09-11 13:04:04.964450306 +0000 UTC m=+498.904151634"
Sep 11 13:04:14.916328 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:14.916296    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-9tmxz"]
Sep 11 13:04:14.916863 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:14.916474    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-9tmxz" podUID="baae8f20-d871-45f4-ac7f-ce1524602050" containerName="limitador" containerID="cri-o://ac7f40134f170e7b17a0c3461cda6f0a972b7bbe17c1cdb4f61436dd0af57a08" gracePeriod=30
Sep 11 13:04:14.917187 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:14.917159    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-9tmxz"
Sep 11 13:04:15.460131 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.460105    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-9tmxz"
Sep 11 13:04:15.616546 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.616464    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/baae8f20-d871-45f4-ac7f-ce1524602050-config-file\") pod \"baae8f20-d871-45f4-ac7f-ce1524602050\" (UID: \"baae8f20-d871-45f4-ac7f-ce1524602050\") "
Sep 11 13:04:15.616546 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.616508    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88w5f\" (UniqueName: \"kubernetes.io/projected/baae8f20-d871-45f4-ac7f-ce1524602050-kube-api-access-88w5f\") pod \"baae8f20-d871-45f4-ac7f-ce1524602050\" (UID: \"baae8f20-d871-45f4-ac7f-ce1524602050\") "
Sep 11 13:04:15.616850 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.616825    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baae8f20-d871-45f4-ac7f-ce1524602050-config-file" (OuterVolumeSpecName: "config-file") pod "baae8f20-d871-45f4-ac7f-ce1524602050" (UID: "baae8f20-d871-45f4-ac7f-ce1524602050"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:04:15.618400 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.618380    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baae8f20-d871-45f4-ac7f-ce1524602050-kube-api-access-88w5f" (OuterVolumeSpecName: "kube-api-access-88w5f") pod "baae8f20-d871-45f4-ac7f-ce1524602050" (UID: "baae8f20-d871-45f4-ac7f-ce1524602050"). InnerVolumeSpecName "kube-api-access-88w5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:04:15.717986 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.717959    2582 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/baae8f20-d871-45f4-ac7f-ce1524602050-config-file\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:04:15.717986 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.717984    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88w5f\" (UniqueName: \"kubernetes.io/projected/baae8f20-d871-45f4-ac7f-ce1524602050-kube-api-access-88w5f\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:04:15.950597 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.950507    2582 generic.go:356] "Generic (PLEG): container finished" podID="baae8f20-d871-45f4-ac7f-ce1524602050" containerID="ac7f40134f170e7b17a0c3461cda6f0a972b7bbe17c1cdb4f61436dd0af57a08" exitCode=0
Sep 11 13:04:15.950597 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.950554    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-9tmxz"
Sep 11 13:04:15.951073 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.950599    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-9tmxz" event={"ID":"baae8f20-d871-45f4-ac7f-ce1524602050","Type":"ContainerDied","Data":"ac7f40134f170e7b17a0c3461cda6f0a972b7bbe17c1cdb4f61436dd0af57a08"}
Sep 11 13:04:15.951073 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.950664    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-9tmxz" event={"ID":"baae8f20-d871-45f4-ac7f-ce1524602050","Type":"ContainerDied","Data":"c0af59cfd621cf0b6e74c68cc27300dfa6e350b0e3ea9400026367aa68aa7842"}
Sep 11 13:04:15.951073 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.950696    2582 scope.go:117] "RemoveContainer" containerID="ac7f40134f170e7b17a0c3461cda6f0a972b7bbe17c1cdb4f61436dd0af57a08"
Sep 11 13:04:15.959794 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.959772    2582 scope.go:117] "RemoveContainer" containerID="ac7f40134f170e7b17a0c3461cda6f0a972b7bbe17c1cdb4f61436dd0af57a08"
Sep 11 13:04:15.960076 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:04:15.960058    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac7f40134f170e7b17a0c3461cda6f0a972b7bbe17c1cdb4f61436dd0af57a08\": container with ID starting with ac7f40134f170e7b17a0c3461cda6f0a972b7bbe17c1cdb4f61436dd0af57a08 not found: ID does not exist" containerID="ac7f40134f170e7b17a0c3461cda6f0a972b7bbe17c1cdb4f61436dd0af57a08"
Sep 11 13:04:15.960133 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.960084    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac7f40134f170e7b17a0c3461cda6f0a972b7bbe17c1cdb4f61436dd0af57a08"} err="failed to get container status \"ac7f40134f170e7b17a0c3461cda6f0a972b7bbe17c1cdb4f61436dd0af57a08\": rpc error: code = NotFound desc = could not find container \"ac7f40134f170e7b17a0c3461cda6f0a972b7bbe17c1cdb4f61436dd0af57a08\": container with ID starting with ac7f40134f170e7b17a0c3461cda6f0a972b7bbe17c1cdb4f61436dd0af57a08 not found: ID does not exist"
Sep 11 13:04:15.972093 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.972072    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-9tmxz"]
Sep 11 13:04:15.974853 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:15.974831    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-9tmxz"]
Sep 11 13:04:16.475969 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:16.475936    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baae8f20-d871-45f4-ac7f-ce1524602050" path="/var/lib/kubelet/pods/baae8f20-d871-45f4-ac7f-ce1524602050/volumes"
Sep 11 13:04:17.700835 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.700804    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f"]
Sep 11 13:04:17.701394 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.701371    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baae8f20-d871-45f4-ac7f-ce1524602050" containerName="limitador"
Sep 11 13:04:17.701501 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.701396    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="baae8f20-d871-45f4-ac7f-ce1524602050" containerName="limitador"
Sep 11 13:04:17.701557 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.701536    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="baae8f20-d871-45f4-ac7f-ce1524602050" containerName="limitador"
Sep 11 13:04:17.705839 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.705816    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f"
Sep 11 13:04:17.708723 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.708704    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 13:04:17.708841 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.708741    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 11 13:04:17.709576 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.709557    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:04:17.709668 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.709557    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-8mm28\""
Sep 11 13:04:17.715141 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.715117    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f"]
Sep 11 13:04:17.835005 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.834960    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/36f77989-5116-42ab-832e-ee5fc4eb42ac-data\") pod \"postgres-69fb4594cf-8sm8f\" (UID: \"36f77989-5116-42ab-832e-ee5fc4eb42ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f"
Sep 11 13:04:17.835155 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.835034    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqv9j\" (UniqueName: \"kubernetes.io/projected/36f77989-5116-42ab-832e-ee5fc4eb42ac-kube-api-access-tqv9j\") pod \"postgres-69fb4594cf-8sm8f\" (UID: \"36f77989-5116-42ab-832e-ee5fc4eb42ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f"
Sep 11 13:04:17.936231 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.936198    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/36f77989-5116-42ab-832e-ee5fc4eb42ac-data\") pod \"postgres-69fb4594cf-8sm8f\" (UID: \"36f77989-5116-42ab-832e-ee5fc4eb42ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f"
Sep 11 13:04:17.936383 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.936279    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqv9j\" (UniqueName: \"kubernetes.io/projected/36f77989-5116-42ab-832e-ee5fc4eb42ac-kube-api-access-tqv9j\") pod \"postgres-69fb4594cf-8sm8f\" (UID: \"36f77989-5116-42ab-832e-ee5fc4eb42ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f"
Sep 11 13:04:17.936554 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.936537    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/36f77989-5116-42ab-832e-ee5fc4eb42ac-data\") pod \"postgres-69fb4594cf-8sm8f\" (UID: \"36f77989-5116-42ab-832e-ee5fc4eb42ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f"
Sep 11 13:04:17.945044 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:17.945024    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqv9j\" (UniqueName: \"kubernetes.io/projected/36f77989-5116-42ab-832e-ee5fc4eb42ac-kube-api-access-tqv9j\") pod \"postgres-69fb4594cf-8sm8f\" (UID: \"36f77989-5116-42ab-832e-ee5fc4eb42ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f"
Sep 11 13:04:18.016578 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:18.016556    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f"
Sep 11 13:04:18.286491 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:18.286388    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f"]
Sep 11 13:04:18.289292 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:04:18.289263    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f77989_5116_42ab_832e_ee5fc4eb42ac.slice/crio-2ec67c7a467951391c2602c24ce5553605d0220b21fd2bf6f6db1ebc1c2081df WatchSource:0}: Error finding container 2ec67c7a467951391c2602c24ce5553605d0220b21fd2bf6f6db1ebc1c2081df: Status 404 returned error can't find the container with id 2ec67c7a467951391c2602c24ce5553605d0220b21fd2bf6f6db1ebc1c2081df
Sep 11 13:04:18.966066 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:18.966027    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f" event={"ID":"36f77989-5116-42ab-832e-ee5fc4eb42ac","Type":"ContainerStarted","Data":"2ec67c7a467951391c2602c24ce5553605d0220b21fd2bf6f6db1ebc1c2081df"}
Sep 11 13:04:24.002409 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:24.002371    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f" event={"ID":"36f77989-5116-42ab-832e-ee5fc4eb42ac","Type":"ContainerStarted","Data":"744f67c79ed43dc583e28a55af2a21ebab5c787427976201ecbcc9cc28211f64"}
Sep 11 13:04:24.002819 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:24.002496    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f"
Sep 11 13:04:24.021905 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:24.021857    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f" podStartSLOduration=2.294853128 podStartE2EDuration="7.021842396s" totalImagesPullingTime="4.726989268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:24.020240557 +0000 UTC m=+517.959941889" watchObservedRunningTime="2026-09-11 13:04:24.021842396 +0000 UTC m=+517.961543729"
Sep 11 13:04:30.035617 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:30.035584    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8sm8f"
Sep 11 13:04:30.678979 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:30.678946    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-plrcs"]
Sep 11 13:04:30.684988 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:30.684957    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-plrcs"
Sep 11 13:04:30.721245 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:30.721214    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-plrcs"]
Sep 11 13:04:30.753486 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:30.753442    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wrcc\" (UniqueName: \"kubernetes.io/projected/1dea9390-edaa-4df4-af0d-e33e40e48105-kube-api-access-8wrcc\") pod \"authorino-557f876f69-plrcs\" (UID: \"1dea9390-edaa-4df4-af0d-e33e40e48105\") " pod="kuadrant-system/authorino-557f876f69-plrcs"
Sep 11 13:04:30.844253 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:30.844214    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-plrcs"]
Sep 11 13:04:30.844536 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:04:30.844516    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-8wrcc], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-plrcs" podUID="1dea9390-edaa-4df4-af0d-e33e40e48105"
Sep 11 13:04:30.854248 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:30.854223    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wrcc\" (UniqueName: \"kubernetes.io/projected/1dea9390-edaa-4df4-af0d-e33e40e48105-kube-api-access-8wrcc\") pod \"authorino-557f876f69-plrcs\" (UID: \"1dea9390-edaa-4df4-af0d-e33e40e48105\") " pod="kuadrant-system/authorino-557f876f69-plrcs"
Sep 11 13:04:30.881334 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:30.881298    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wrcc\" (UniqueName: \"kubernetes.io/projected/1dea9390-edaa-4df4-af0d-e33e40e48105-kube-api-access-8wrcc\") pod \"authorino-557f876f69-plrcs\" (UID: \"1dea9390-edaa-4df4-af0d-e33e40e48105\") " pod="kuadrant-system/authorino-557f876f69-plrcs"
Sep 11 13:04:30.905318 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:30.905290    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-tmb5n"]
Sep 11 13:04:30.909641 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:30.909621    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-tmb5n"
Sep 11 13:04:30.913842 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:30.913825    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 13:04:30.925449 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:30.925410    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-tmb5n"]
Sep 11 13:04:30.954683 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:30.954613    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62-tls-cert\") pod \"authorino-6d4dfc649f-tmb5n\" (UID: \"3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62\") " pod="kuadrant-system/authorino-6d4dfc649f-tmb5n"
Sep 11 13:04:30.954683 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:30.954643    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phvtf\" (UniqueName: \"kubernetes.io/projected/3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62-kube-api-access-phvtf\") pod \"authorino-6d4dfc649f-tmb5n\" (UID: \"3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62\") " pod="kuadrant-system/authorino-6d4dfc649f-tmb5n"
Sep 11 13:04:31.025168 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.025138    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-tmb5n"]
Sep 11 13:04:31.025452 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:04:31.025413    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-phvtf tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-tmb5n" podUID="3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62"
Sep 11 13:04:31.046160 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.046129    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-tmb5n"
Sep 11 13:04:31.046523 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.046132    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-plrcs"
Sep 11 13:04:31.053731 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.053710    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-tmb5n"
Sep 11 13:04:31.055756 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.055733    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phvtf\" (UniqueName: \"kubernetes.io/projected/3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62-kube-api-access-phvtf\") pod \"authorino-6d4dfc649f-tmb5n\" (UID: \"3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62\") " pod="kuadrant-system/authorino-6d4dfc649f-tmb5n"
Sep 11 13:04:31.055872 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.055821    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62-tls-cert\") pod \"authorino-6d4dfc649f-tmb5n\" (UID: \"3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62\") " pod="kuadrant-system/authorino-6d4dfc649f-tmb5n"
Sep 11 13:04:31.057063 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.057043    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-plrcs"
Sep 11 13:04:31.058281 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.058255    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62-tls-cert\") pod \"authorino-6d4dfc649f-tmb5n\" (UID: \"3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62\") " pod="kuadrant-system/authorino-6d4dfc649f-tmb5n"
Sep 11 13:04:31.079400 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.079368    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phvtf\" (UniqueName: \"kubernetes.io/projected/3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62-kube-api-access-phvtf\") pod \"authorino-6d4dfc649f-tmb5n\" (UID: \"3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62\") " pod="kuadrant-system/authorino-6d4dfc649f-tmb5n"
Sep 11 13:04:31.156098 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.156060    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62-tls-cert\") pod \"3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62\" (UID: \"3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62\") "
Sep 11 13:04:31.156268 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.156162    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wrcc\" (UniqueName: \"kubernetes.io/projected/1dea9390-edaa-4df4-af0d-e33e40e48105-kube-api-access-8wrcc\") pod \"1dea9390-edaa-4df4-af0d-e33e40e48105\" (UID: \"1dea9390-edaa-4df4-af0d-e33e40e48105\") "
Sep 11 13:04:31.156268 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.156178    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phvtf\" (UniqueName: \"kubernetes.io/projected/3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62-kube-api-access-phvtf\") pod \"3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62\" (UID: \"3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62\") "
Sep 11 13:04:31.158221 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.158186    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62" (UID: "3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:04:31.158221 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.158207    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62-kube-api-access-phvtf" (OuterVolumeSpecName: "kube-api-access-phvtf") pod "3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62" (UID: "3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62"). InnerVolumeSpecName "kube-api-access-phvtf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:04:31.158363 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.158269    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dea9390-edaa-4df4-af0d-e33e40e48105-kube-api-access-8wrcc" (OuterVolumeSpecName: "kube-api-access-8wrcc") pod "1dea9390-edaa-4df4-af0d-e33e40e48105" (UID: "1dea9390-edaa-4df4-af0d-e33e40e48105"). InnerVolumeSpecName "kube-api-access-8wrcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:04:31.257462 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.257432    2582 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62-tls-cert\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:04:31.257462 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.257457    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wrcc\" (UniqueName: \"kubernetes.io/projected/1dea9390-edaa-4df4-af0d-e33e40e48105-kube-api-access-8wrcc\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:04:31.257462 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:31.257467    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phvtf\" (UniqueName: \"kubernetes.io/projected/3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62-kube-api-access-phvtf\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:04:32.051091 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:32.051062    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-tmb5n"
Sep 11 13:04:32.051488 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:32.051062    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-plrcs"
Sep 11 13:04:32.135496 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:32.135460    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-tmb5n"]
Sep 11 13:04:32.140220 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:32.140193    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-tmb5n"]
Sep 11 13:04:32.191038 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:32.191003    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-plrcs"]
Sep 11 13:04:32.197117 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:32.197090    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-plrcs"]
Sep 11 13:04:32.475990 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:32.475960    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dea9390-edaa-4df4-af0d-e33e40e48105" path="/var/lib/kubelet/pods/1dea9390-edaa-4df4-af0d-e33e40e48105/volumes"
Sep 11 13:04:32.476227 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:32.476215    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62" path="/var/lib/kubelet/pods/3c26d9d4-c37e-4fc4-905f-cd0ab8dfee62/volumes"
Sep 11 13:04:33.390118 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:33.390083    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-pn7rp"]
Sep 11 13:04:33.390568 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:33.390292    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-pn7rp" podUID="183e9f9a-f96c-4601-8384-c0a77d5074fe" containerName="authorino" containerID="cri-o://1b2b5fc9eb7bd68c396b3a9710c29c069f30ec47998d288088dbbdb0c2e41861" gracePeriod=30
Sep 11 13:04:33.639683 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:33.639663    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-pn7rp"
Sep 11 13:04:33.676299 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:33.676224    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znkj7\" (UniqueName: \"kubernetes.io/projected/183e9f9a-f96c-4601-8384-c0a77d5074fe-kube-api-access-znkj7\") pod \"183e9f9a-f96c-4601-8384-c0a77d5074fe\" (UID: \"183e9f9a-f96c-4601-8384-c0a77d5074fe\") "
Sep 11 13:04:33.678198 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:33.678173    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/183e9f9a-f96c-4601-8384-c0a77d5074fe-kube-api-access-znkj7" (OuterVolumeSpecName: "kube-api-access-znkj7") pod "183e9f9a-f96c-4601-8384-c0a77d5074fe" (UID: "183e9f9a-f96c-4601-8384-c0a77d5074fe"). InnerVolumeSpecName "kube-api-access-znkj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:04:33.777141 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:33.777107    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znkj7\" (UniqueName: \"kubernetes.io/projected/183e9f9a-f96c-4601-8384-c0a77d5074fe-kube-api-access-znkj7\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:04:34.062891 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:34.062849    2582 generic.go:356] "Generic (PLEG): container finished" podID="183e9f9a-f96c-4601-8384-c0a77d5074fe" containerID="1b2b5fc9eb7bd68c396b3a9710c29c069f30ec47998d288088dbbdb0c2e41861" exitCode=0
Sep 11 13:04:34.063063 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:34.062898    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-pn7rp"
Sep 11 13:04:34.063063 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:34.062941    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-pn7rp" event={"ID":"183e9f9a-f96c-4601-8384-c0a77d5074fe","Type":"ContainerDied","Data":"1b2b5fc9eb7bd68c396b3a9710c29c069f30ec47998d288088dbbdb0c2e41861"}
Sep 11 13:04:34.063063 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:34.062991    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-pn7rp" event={"ID":"183e9f9a-f96c-4601-8384-c0a77d5074fe","Type":"ContainerDied","Data":"445f415b76ce99683ea2f20a5cb755b040d55e7e6ae06e1249a3dbd5ffc2260e"}
Sep 11 13:04:34.063063 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:34.063014    2582 scope.go:117] "RemoveContainer" containerID="1b2b5fc9eb7bd68c396b3a9710c29c069f30ec47998d288088dbbdb0c2e41861"
Sep 11 13:04:34.071950 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:34.071932    2582 scope.go:117] "RemoveContainer" containerID="1b2b5fc9eb7bd68c396b3a9710c29c069f30ec47998d288088dbbdb0c2e41861"
Sep 11 13:04:34.072258 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:04:34.072231    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b2b5fc9eb7bd68c396b3a9710c29c069f30ec47998d288088dbbdb0c2e41861\": container with ID starting with 1b2b5fc9eb7bd68c396b3a9710c29c069f30ec47998d288088dbbdb0c2e41861 not found: ID does not exist" containerID="1b2b5fc9eb7bd68c396b3a9710c29c069f30ec47998d288088dbbdb0c2e41861"
Sep 11 13:04:34.072363 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:34.072269    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b2b5fc9eb7bd68c396b3a9710c29c069f30ec47998d288088dbbdb0c2e41861"} err="failed to get container status \"1b2b5fc9eb7bd68c396b3a9710c29c069f30ec47998d288088dbbdb0c2e41861\": rpc error: code = NotFound desc = could not find container \"1b2b5fc9eb7bd68c396b3a9710c29c069f30ec47998d288088dbbdb0c2e41861\": container with ID starting with 1b2b5fc9eb7bd68c396b3a9710c29c069f30ec47998d288088dbbdb0c2e41861 not found: ID does not exist"
Sep 11 13:04:34.108979 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:34.108950    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-pn7rp"]
Sep 11 13:04:34.117741 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:34.117715    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-pn7rp"]
Sep 11 13:04:34.476321 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:34.476293    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="183e9f9a-f96c-4601-8384-c0a77d5074fe" path="/var/lib/kubelet/pods/183e9f9a-f96c-4601-8384-c0a77d5074fe/volumes"
Sep 11 13:04:39.141099 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.141019    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-574555cdc6-zpqhw"]
Sep 11 13:04:39.141575 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.141407    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="183e9f9a-f96c-4601-8384-c0a77d5074fe" containerName="authorino"
Sep 11 13:04:39.141575 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.141430    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="183e9f9a-f96c-4601-8384-c0a77d5074fe" containerName="authorino"
Sep 11 13:04:39.141575 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.141499    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="183e9f9a-f96c-4601-8384-c0a77d5074fe" containerName="authorino"
Sep 11 13:04:39.159933 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.159902    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-574555cdc6-zpqhw"
Sep 11 13:04:39.166593 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.166566    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 13:04:39.166724 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.166698    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 13:04:39.166864 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.166843    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dmcj8\""
Sep 11 13:04:39.167958 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.167632    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 13:04:39.198479 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.198448    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-574555cdc6-zpqhw"]
Sep 11 13:04:39.229237 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.229198    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27c9b580-eb86-4a3e-a783-f98b85fe0e19-webhook-cert\") pod \"maas-controller-574555cdc6-zpqhw\" (UID: \"27c9b580-eb86-4a3e-a783-f98b85fe0e19\") " pod="opendatahub/maas-controller-574555cdc6-zpqhw"
Sep 11 13:04:39.229434 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.229243    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq5xb\" (UniqueName: \"kubernetes.io/projected/27c9b580-eb86-4a3e-a783-f98b85fe0e19-kube-api-access-fq5xb\") pod \"maas-controller-574555cdc6-zpqhw\" (UID: \"27c9b580-eb86-4a3e-a783-f98b85fe0e19\") " pod="opendatahub/maas-controller-574555cdc6-zpqhw"
Sep 11 13:04:39.229434 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.229274    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27c9b580-eb86-4a3e-a783-f98b85fe0e19-metrics-certs\") pod \"maas-controller-574555cdc6-zpqhw\" (UID: \"27c9b580-eb86-4a3e-a783-f98b85fe0e19\") " pod="opendatahub/maas-controller-574555cdc6-zpqhw"
Sep 11 13:04:39.329919 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.329879    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27c9b580-eb86-4a3e-a783-f98b85fe0e19-metrics-certs\") pod \"maas-controller-574555cdc6-zpqhw\" (UID: \"27c9b580-eb86-4a3e-a783-f98b85fe0e19\") " pod="opendatahub/maas-controller-574555cdc6-zpqhw"
Sep 11 13:04:39.330090 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.329962    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27c9b580-eb86-4a3e-a783-f98b85fe0e19-webhook-cert\") pod \"maas-controller-574555cdc6-zpqhw\" (UID: \"27c9b580-eb86-4a3e-a783-f98b85fe0e19\") " pod="opendatahub/maas-controller-574555cdc6-zpqhw"
Sep 11 13:04:39.330090 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.329991    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq5xb\" (UniqueName: \"kubernetes.io/projected/27c9b580-eb86-4a3e-a783-f98b85fe0e19-kube-api-access-fq5xb\") pod \"maas-controller-574555cdc6-zpqhw\" (UID: \"27c9b580-eb86-4a3e-a783-f98b85fe0e19\") " pod="opendatahub/maas-controller-574555cdc6-zpqhw"
Sep 11 13:04:39.332455 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.332406    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27c9b580-eb86-4a3e-a783-f98b85fe0e19-webhook-cert\") pod \"maas-controller-574555cdc6-zpqhw\" (UID: \"27c9b580-eb86-4a3e-a783-f98b85fe0e19\") " pod="opendatahub/maas-controller-574555cdc6-zpqhw"
Sep 11 13:04:39.332557 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.332486    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27c9b580-eb86-4a3e-a783-f98b85fe0e19-metrics-certs\") pod \"maas-controller-574555cdc6-zpqhw\" (UID: \"27c9b580-eb86-4a3e-a783-f98b85fe0e19\") " pod="opendatahub/maas-controller-574555cdc6-zpqhw"
Sep 11 13:04:39.357655 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.357609    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq5xb\" (UniqueName: \"kubernetes.io/projected/27c9b580-eb86-4a3e-a783-f98b85fe0e19-kube-api-access-fq5xb\") pod \"maas-controller-574555cdc6-zpqhw\" (UID: \"27c9b580-eb86-4a3e-a783-f98b85fe0e19\") " pod="opendatahub/maas-controller-574555cdc6-zpqhw"
Sep 11 13:04:39.426662 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.426581    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65b89bb8c4-d7jx8"]
Sep 11 13:04:39.430087 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.430070    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65b89bb8c4-d7jx8"
Sep 11 13:04:39.471917 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.471884    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-574555cdc6-zpqhw"
Sep 11 13:04:39.477457 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.477429    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65b89bb8c4-d7jx8"]
Sep 11 13:04:39.532818 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.532791    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgwqb\" (UniqueName: \"kubernetes.io/projected/8c2abef0-5a85-40d5-8fec-9db765343ceb-kube-api-access-sgwqb\") pod \"maas-controller-65b89bb8c4-d7jx8\" (UID: \"8c2abef0-5a85-40d5-8fec-9db765343ceb\") " pod="opendatahub/maas-controller-65b89bb8c4-d7jx8"
Sep 11 13:04:39.532949 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.532920    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c2abef0-5a85-40d5-8fec-9db765343ceb-metrics-certs\") pod \"maas-controller-65b89bb8c4-d7jx8\" (UID: \"8c2abef0-5a85-40d5-8fec-9db765343ceb\") " pod="opendatahub/maas-controller-65b89bb8c4-d7jx8"
Sep 11 13:04:39.533070 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.533033    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c2abef0-5a85-40d5-8fec-9db765343ceb-webhook-cert\") pod \"maas-controller-65b89bb8c4-d7jx8\" (UID: \"8c2abef0-5a85-40d5-8fec-9db765343ceb\") " pod="opendatahub/maas-controller-65b89bb8c4-d7jx8"
Sep 11 13:04:39.633770 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.633729    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgwqb\" (UniqueName: \"kubernetes.io/projected/8c2abef0-5a85-40d5-8fec-9db765343ceb-kube-api-access-sgwqb\") pod \"maas-controller-65b89bb8c4-d7jx8\" (UID: \"8c2abef0-5a85-40d5-8fec-9db765343ceb\") " pod="opendatahub/maas-controller-65b89bb8c4-d7jx8"
Sep 11 13:04:39.633770 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.633775    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c2abef0-5a85-40d5-8fec-9db765343ceb-metrics-certs\") pod \"maas-controller-65b89bb8c4-d7jx8\" (UID: \"8c2abef0-5a85-40d5-8fec-9db765343ceb\") " pod="opendatahub/maas-controller-65b89bb8c4-d7jx8"
Sep 11 13:04:39.634013 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.633905    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c2abef0-5a85-40d5-8fec-9db765343ceb-webhook-cert\") pod \"maas-controller-65b89bb8c4-d7jx8\" (UID: \"8c2abef0-5a85-40d5-8fec-9db765343ceb\") " pod="opendatahub/maas-controller-65b89bb8c4-d7jx8"
Sep 11 13:04:39.636226 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.636191    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c2abef0-5a85-40d5-8fec-9db765343ceb-webhook-cert\") pod \"maas-controller-65b89bb8c4-d7jx8\" (UID: \"8c2abef0-5a85-40d5-8fec-9db765343ceb\") " pod="opendatahub/maas-controller-65b89bb8c4-d7jx8"
Sep 11 13:04:39.636226 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.636209    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c2abef0-5a85-40d5-8fec-9db765343ceb-metrics-certs\") pod \"maas-controller-65b89bb8c4-d7jx8\" (UID: \"8c2abef0-5a85-40d5-8fec-9db765343ceb\") " pod="opendatahub/maas-controller-65b89bb8c4-d7jx8"
Sep 11 13:04:39.645702 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.645676    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgwqb\" (UniqueName: \"kubernetes.io/projected/8c2abef0-5a85-40d5-8fec-9db765343ceb-kube-api-access-sgwqb\") pod \"maas-controller-65b89bb8c4-d7jx8\" (UID: \"8c2abef0-5a85-40d5-8fec-9db765343ceb\") " pod="opendatahub/maas-controller-65b89bb8c4-d7jx8"
Sep 11 13:04:39.740976 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.740942    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65b89bb8c4-d7jx8"
Sep 11 13:04:39.792790 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:39.792758    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-574555cdc6-zpqhw"]
Sep 11 13:04:39.794765 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:04:39.794723    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c9b580_eb86_4a3e_a783_f98b85fe0e19.slice/crio-d2f365d74e814e256268792f6f053a543265e58ab6c88150ffc60c3f75005802 WatchSource:0}: Error finding container d2f365d74e814e256268792f6f053a543265e58ab6c88150ffc60c3f75005802: Status 404 returned error can't find the container with id d2f365d74e814e256268792f6f053a543265e58ab6c88150ffc60c3f75005802
Sep 11 13:04:40.026803 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:40.026726    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65b89bb8c4-d7jx8"]
Sep 11 13:04:40.029682 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:04:40.029655    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c2abef0_5a85_40d5_8fec_9db765343ceb.slice/crio-5533e3b3d5389c21e340d124e06ab8554a521f14ac38b45e66f11889b850da49 WatchSource:0}: Error finding container 5533e3b3d5389c21e340d124e06ab8554a521f14ac38b45e66f11889b850da49: Status 404 returned error can't find the container with id 5533e3b3d5389c21e340d124e06ab8554a521f14ac38b45e66f11889b850da49
Sep 11 13:04:40.093625 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:40.093578    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65b89bb8c4-d7jx8" event={"ID":"8c2abef0-5a85-40d5-8fec-9db765343ceb","Type":"ContainerStarted","Data":"5533e3b3d5389c21e340d124e06ab8554a521f14ac38b45e66f11889b850da49"}
Sep 11 13:04:40.094755 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:40.094731    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-574555cdc6-zpqhw" event={"ID":"27c9b580-eb86-4a3e-a783-f98b85fe0e19","Type":"ContainerStarted","Data":"d2f365d74e814e256268792f6f053a543265e58ab6c88150ffc60c3f75005802"}
Sep 11 13:04:43.115357 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:43.115305    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-574555cdc6-zpqhw" event={"ID":"27c9b580-eb86-4a3e-a783-f98b85fe0e19","Type":"ContainerStarted","Data":"2aa9f8a2b8d4c623266a15f925e7f35c4bad3ce6b9ea3504b4539935e90d21da"}
Sep 11 13:04:43.115866 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:43.115378    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-574555cdc6-zpqhw"
Sep 11 13:04:43.116821 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:43.116799    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65b89bb8c4-d7jx8" event={"ID":"8c2abef0-5a85-40d5-8fec-9db765343ceb","Type":"ContainerStarted","Data":"e069d519d91cccf027660b14a5d3788f0f4558dd506f26b21cc33bae441e2f0d"}
Sep 11 13:04:43.116931 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:43.116919    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65b89bb8c4-d7jx8"
Sep 11 13:04:43.157556 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:43.157507    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-574555cdc6-zpqhw" podStartSLOduration=1.590727376 podStartE2EDuration="4.157491193s" totalImagesPullingTime="2.566763817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:43.156457277 +0000 UTC m=+537.096158609" watchObservedRunningTime="2026-09-11 13:04:43.157491193 +0000 UTC m=+537.097192526"
Sep 11 13:04:43.199633 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:43.199578    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65b89bb8c4-d7jx8" podStartSLOduration=1.859622457 podStartE2EDuration="4.199559985s" totalImagesPullingTime="2.339937528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:43.197897092 +0000 UTC m=+537.137598424" watchObservedRunningTime="2026-09-11 13:04:43.199559985 +0000 UTC m=+537.139261317"
Sep 11 13:04:54.123390 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:54.123352    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-574555cdc6-zpqhw"
Sep 11 13:04:54.123929 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:54.123471    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65b89bb8c4-d7jx8"
Sep 11 13:04:54.200217 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:54.200180    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-574555cdc6-zpqhw"]
Sep 11 13:04:54.200454 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:54.200390    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-574555cdc6-zpqhw" podUID="27c9b580-eb86-4a3e-a783-f98b85fe0e19" containerName="manager" containerID="cri-o://2aa9f8a2b8d4c623266a15f925e7f35c4bad3ce6b9ea3504b4539935e90d21da" gracePeriod=30
Sep 11 13:04:54.448153 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:54.448123    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-574555cdc6-zpqhw"
Sep 11 13:04:54.583243 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:54.583201    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fq5xb\" (UniqueName: \"kubernetes.io/projected/27c9b580-eb86-4a3e-a783-f98b85fe0e19-kube-api-access-fq5xb\") pod \"27c9b580-eb86-4a3e-a783-f98b85fe0e19\" (UID: \"27c9b580-eb86-4a3e-a783-f98b85fe0e19\") "
Sep 11 13:04:54.583457 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:54.583277    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27c9b580-eb86-4a3e-a783-f98b85fe0e19-metrics-certs\") pod \"27c9b580-eb86-4a3e-a783-f98b85fe0e19\" (UID: \"27c9b580-eb86-4a3e-a783-f98b85fe0e19\") "
Sep 11 13:04:54.583457 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:54.583367    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27c9b580-eb86-4a3e-a783-f98b85fe0e19-webhook-cert\") pod \"27c9b580-eb86-4a3e-a783-f98b85fe0e19\" (UID: \"27c9b580-eb86-4a3e-a783-f98b85fe0e19\") "
Sep 11 13:04:54.585407 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:54.585371    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27c9b580-eb86-4a3e-a783-f98b85fe0e19-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "27c9b580-eb86-4a3e-a783-f98b85fe0e19" (UID: "27c9b580-eb86-4a3e-a783-f98b85fe0e19"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:04:54.585556 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:54.585455    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27c9b580-eb86-4a3e-a783-f98b85fe0e19-kube-api-access-fq5xb" (OuterVolumeSpecName: "kube-api-access-fq5xb") pod "27c9b580-eb86-4a3e-a783-f98b85fe0e19" (UID: "27c9b580-eb86-4a3e-a783-f98b85fe0e19"). InnerVolumeSpecName "kube-api-access-fq5xb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:04:54.585556 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:54.585510    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27c9b580-eb86-4a3e-a783-f98b85fe0e19-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "27c9b580-eb86-4a3e-a783-f98b85fe0e19" (UID: "27c9b580-eb86-4a3e-a783-f98b85fe0e19"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:04:54.684742 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:54.684642    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27c9b580-eb86-4a3e-a783-f98b85fe0e19-webhook-cert\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:04:54.684742 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:54.684676    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fq5xb\" (UniqueName: \"kubernetes.io/projected/27c9b580-eb86-4a3e-a783-f98b85fe0e19-kube-api-access-fq5xb\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:04:54.684742 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:54.684690    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27c9b580-eb86-4a3e-a783-f98b85fe0e19-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:04:55.179979 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:55.179948    2582 generic.go:356] "Generic (PLEG): container finished" podID="27c9b580-eb86-4a3e-a783-f98b85fe0e19" containerID="2aa9f8a2b8d4c623266a15f925e7f35c4bad3ce6b9ea3504b4539935e90d21da" exitCode=0
Sep 11 13:04:55.180443 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:55.179988    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-574555cdc6-zpqhw" event={"ID":"27c9b580-eb86-4a3e-a783-f98b85fe0e19","Type":"ContainerDied","Data":"2aa9f8a2b8d4c623266a15f925e7f35c4bad3ce6b9ea3504b4539935e90d21da"}
Sep 11 13:04:55.180443 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:55.179994    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-574555cdc6-zpqhw"
Sep 11 13:04:55.180443 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:55.180027    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-574555cdc6-zpqhw" event={"ID":"27c9b580-eb86-4a3e-a783-f98b85fe0e19","Type":"ContainerDied","Data":"d2f365d74e814e256268792f6f053a543265e58ab6c88150ffc60c3f75005802"}
Sep 11 13:04:55.180443 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:55.180050    2582 scope.go:117] "RemoveContainer" containerID="2aa9f8a2b8d4c623266a15f925e7f35c4bad3ce6b9ea3504b4539935e90d21da"
Sep 11 13:04:55.189179 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:55.189162    2582 scope.go:117] "RemoveContainer" containerID="2aa9f8a2b8d4c623266a15f925e7f35c4bad3ce6b9ea3504b4539935e90d21da"
Sep 11 13:04:55.189484 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:04:55.189467    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aa9f8a2b8d4c623266a15f925e7f35c4bad3ce6b9ea3504b4539935e90d21da\": container with ID starting with 2aa9f8a2b8d4c623266a15f925e7f35c4bad3ce6b9ea3504b4539935e90d21da not found: ID does not exist" containerID="2aa9f8a2b8d4c623266a15f925e7f35c4bad3ce6b9ea3504b4539935e90d21da"
Sep 11 13:04:55.189538 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:55.189493    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aa9f8a2b8d4c623266a15f925e7f35c4bad3ce6b9ea3504b4539935e90d21da"} err="failed to get container status \"2aa9f8a2b8d4c623266a15f925e7f35c4bad3ce6b9ea3504b4539935e90d21da\": rpc error: code = NotFound desc = could not find container \"2aa9f8a2b8d4c623266a15f925e7f35c4bad3ce6b9ea3504b4539935e90d21da\": container with ID starting with 2aa9f8a2b8d4c623266a15f925e7f35c4bad3ce6b9ea3504b4539935e90d21da not found: ID does not exist"
Sep 11 13:04:55.205944 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:55.205912    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-574555cdc6-zpqhw"]
Sep 11 13:04:55.213173 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:55.213147    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-574555cdc6-zpqhw"]
Sep 11 13:04:56.476163 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:04:56.476121    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27c9b580-eb86-4a3e-a783-f98b85fe0e19" path="/var/lib/kubelet/pods/27c9b580-eb86-4a3e-a783-f98b85fe0e19/volumes"
Sep 11 13:05:17.110770 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.110733    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-78554b866-tht9f"]
Sep 11 13:05:17.111218 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.111140    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27c9b580-eb86-4a3e-a783-f98b85fe0e19" containerName="manager"
Sep 11 13:05:17.111218 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.111151    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="27c9b580-eb86-4a3e-a783-f98b85fe0e19" containerName="manager"
Sep 11 13:05:17.111291 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.111221    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="27c9b580-eb86-4a3e-a783-f98b85fe0e19" containerName="manager"
Sep 11 13:05:17.117670 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.117651    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:17.121052 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.121033    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kb4hz\""
Sep 11 13:05:17.121185 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.121163    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 13:05:17.121983 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.121968    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 13:05:17.135575 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.135547    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-78554b866-tht9f"]
Sep 11 13:05:17.175359 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.175329    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb898b-d767-4ab3-8b74-b8210565ffa4-metrics-certs\") pod \"maas-api-78554b866-tht9f\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") " pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:17.175359 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.175362    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fefb898b-d767-4ab3-8b74-b8210565ffa4-openshift-service-ca\") pod \"maas-api-78554b866-tht9f\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") " pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:17.175575 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.175380    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m88wt\" (UniqueName: \"kubernetes.io/projected/fefb898b-d767-4ab3-8b74-b8210565ffa4-kube-api-access-m88wt\") pod \"maas-api-78554b866-tht9f\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") " pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:17.175575 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.175460    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fefb898b-d767-4ab3-8b74-b8210565ffa4-maas-api-tls\") pod \"maas-api-78554b866-tht9f\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") " pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:17.275881 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.275845    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb898b-d767-4ab3-8b74-b8210565ffa4-metrics-certs\") pod \"maas-api-78554b866-tht9f\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") " pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:17.276047 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.275889    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fefb898b-d767-4ab3-8b74-b8210565ffa4-openshift-service-ca\") pod \"maas-api-78554b866-tht9f\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") " pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:17.276047 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.275912    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m88wt\" (UniqueName: \"kubernetes.io/projected/fefb898b-d767-4ab3-8b74-b8210565ffa4-kube-api-access-m88wt\") pod \"maas-api-78554b866-tht9f\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") " pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:17.276047 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.275955    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fefb898b-d767-4ab3-8b74-b8210565ffa4-maas-api-tls\") pod \"maas-api-78554b866-tht9f\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") " pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:17.276047 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:05:17.275986    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 11 13:05:17.276288 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:05:17.276078    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fefb898b-d767-4ab3-8b74-b8210565ffa4-metrics-certs podName:fefb898b-d767-4ab3-8b74-b8210565ffa4 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:17.776055962 +0000 UTC m=+571.715757272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fefb898b-d767-4ab3-8b74-b8210565ffa4-metrics-certs") pod "maas-api-78554b866-tht9f" (UID: "fefb898b-d767-4ab3-8b74-b8210565ffa4") : secret "maas-api-metrics-tls" not found
Sep 11 13:05:17.276705 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.276683    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fefb898b-d767-4ab3-8b74-b8210565ffa4-openshift-service-ca\") pod \"maas-api-78554b866-tht9f\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") " pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:17.278354 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.278332    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fefb898b-d767-4ab3-8b74-b8210565ffa4-maas-api-tls\") pod \"maas-api-78554b866-tht9f\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") " pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:17.289291 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.289271    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m88wt\" (UniqueName: \"kubernetes.io/projected/fefb898b-d767-4ab3-8b74-b8210565ffa4-kube-api-access-m88wt\") pod \"maas-api-78554b866-tht9f\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") " pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:17.780777 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.780741    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb898b-d767-4ab3-8b74-b8210565ffa4-metrics-certs\") pod \"maas-api-78554b866-tht9f\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") " pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:17.783034 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.783011    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb898b-d767-4ab3-8b74-b8210565ffa4-metrics-certs\") pod \"maas-api-78554b866-tht9f\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") " pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:17.916698 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.916659    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6"]
Sep 11 13:05:17.920316 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.920300    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6"
Sep 11 13:05:17.923442 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.923406    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 13:05:17.923646 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.923628    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 13:05:17.924473 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.924456    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-tx4fz\""
Sep 11 13:05:17.924473 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.924467    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 13:05:17.937987 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.937966    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6"]
Sep 11 13:05:17.982061 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.982037    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-jhml6\" (UID: \"fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6"
Sep 11 13:05:17.982163 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:17.982077    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj8cg\" (UniqueName: \"kubernetes.io/projected/fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90-kube-api-access-tj8cg\") pod \"payload-pre-processing-59fbc47bf8-jhml6\" (UID: \"fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6"
Sep 11 13:05:18.006227 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.006202    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-84qbs"]
Sep 11 13:05:18.009459 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.009445    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-84qbs"
Sep 11 13:05:18.027586 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.027566    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:18.033525 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.033463    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-84qbs"]
Sep 11 13:05:18.082566 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.082536    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-jhml6\" (UID: \"fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6"
Sep 11 13:05:18.082717 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.082582    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj8cg\" (UniqueName: \"kubernetes.io/projected/fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90-kube-api-access-tj8cg\") pod \"payload-pre-processing-59fbc47bf8-jhml6\" (UID: \"fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6"
Sep 11 13:05:18.082717 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.082603    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71f3fe4f-d604-49ce-8364-6643e5c1afc1-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-84qbs\" (UID: \"71f3fe4f-d604-49ce-8364-6643e5c1afc1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-84qbs"
Sep 11 13:05:18.082717 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.082651    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjd7k\" (UniqueName: \"kubernetes.io/projected/71f3fe4f-d604-49ce-8364-6643e5c1afc1-kube-api-access-xjd7k\") pod \"payload-processing-5b54b7fbb7-84qbs\" (UID: \"71f3fe4f-d604-49ce-8364-6643e5c1afc1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-84qbs"
Sep 11 13:05:18.083327 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.083306    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-jhml6\" (UID: \"fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6"
Sep 11 13:05:18.103401 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.103375    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj8cg\" (UniqueName: \"kubernetes.io/projected/fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90-kube-api-access-tj8cg\") pod \"payload-pre-processing-59fbc47bf8-jhml6\" (UID: \"fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6"
Sep 11 13:05:18.183170 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.183140    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71f3fe4f-d604-49ce-8364-6643e5c1afc1-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-84qbs\" (UID: \"71f3fe4f-d604-49ce-8364-6643e5c1afc1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-84qbs"
Sep 11 13:05:18.183522 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.183203    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjd7k\" (UniqueName: \"kubernetes.io/projected/71f3fe4f-d604-49ce-8364-6643e5c1afc1-kube-api-access-xjd7k\") pod \"payload-processing-5b54b7fbb7-84qbs\" (UID: \"71f3fe4f-d604-49ce-8364-6643e5c1afc1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-84qbs"
Sep 11 13:05:18.183931 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.183911    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71f3fe4f-d604-49ce-8364-6643e5c1afc1-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-84qbs\" (UID: \"71f3fe4f-d604-49ce-8364-6643e5c1afc1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-84qbs"
Sep 11 13:05:18.203955 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.203925    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjd7k\" (UniqueName: \"kubernetes.io/projected/71f3fe4f-d604-49ce-8364-6643e5c1afc1-kube-api-access-xjd7k\") pod \"payload-processing-5b54b7fbb7-84qbs\" (UID: \"71f3fe4f-d604-49ce-8364-6643e5c1afc1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-84qbs"
Sep 11 13:05:18.231377 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.231333    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6"
Sep 11 13:05:18.319321 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.319251    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-84qbs"
Sep 11 13:05:18.323344 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.323322    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-78554b866-tht9f"]
Sep 11 13:05:18.325977 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:05:18.325956    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfefb898b_d767_4ab3_8b74_b8210565ffa4.slice/crio-a6065ee10138a8e75d8a95614557f81166cfd4d286376a0f09676b9dcb0cca4b WatchSource:0}: Error finding container a6065ee10138a8e75d8a95614557f81166cfd4d286376a0f09676b9dcb0cca4b: Status 404 returned error can't find the container with id a6065ee10138a8e75d8a95614557f81166cfd4d286376a0f09676b9dcb0cca4b
Sep 11 13:05:18.504975 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.504943    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6"]
Sep 11 13:05:18.508371 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:05:18.508336    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa1cd2d7_fe47_45f6_b0ca_e36fb1c44d90.slice/crio-74dc237eee94f337f98994480a821a0f6723f5ffa19618207e3fb56e3e0d492f WatchSource:0}: Error finding container 74dc237eee94f337f98994480a821a0f6723f5ffa19618207e3fb56e3e0d492f: Status 404 returned error can't find the container with id 74dc237eee94f337f98994480a821a0f6723f5ffa19618207e3fb56e3e0d492f
Sep 11 13:05:18.593485 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:18.593460    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-84qbs"]
Sep 11 13:05:18.595688 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:05:18.595661    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f3fe4f_d604_49ce_8364_6643e5c1afc1.slice/crio-52b82322c97b08341848aed2d84556d99b1c75fbe08c11ee799b1adf979b254d WatchSource:0}: Error finding container 52b82322c97b08341848aed2d84556d99b1c75fbe08c11ee799b1adf979b254d: Status 404 returned error can't find the container with id 52b82322c97b08341848aed2d84556d99b1c75fbe08c11ee799b1adf979b254d
Sep 11 13:05:19.308721 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:19.308644    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6" event={"ID":"fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90","Type":"ContainerStarted","Data":"74dc237eee94f337f98994480a821a0f6723f5ffa19618207e3fb56e3e0d492f"}
Sep 11 13:05:19.311337 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:19.311308    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-84qbs" event={"ID":"71f3fe4f-d604-49ce-8364-6643e5c1afc1","Type":"ContainerStarted","Data":"52b82322c97b08341848aed2d84556d99b1c75fbe08c11ee799b1adf979b254d"}
Sep 11 13:05:19.313354 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:19.313282    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f" event={"ID":"fefb898b-d767-4ab3-8b74-b8210565ffa4","Type":"ContainerStarted","Data":"a6065ee10138a8e75d8a95614557f81166cfd4d286376a0f09676b9dcb0cca4b"}
Sep 11 13:05:22.332231 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:22.332197    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6" event={"ID":"fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90","Type":"ContainerStarted","Data":"b935bdb27660227afceddcac0b9a57a4ac292292b860201174ee556a9df66091"}
Sep 11 13:05:22.332694 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:22.332289    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6"
Sep 11 13:05:22.333772 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:22.333747    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-84qbs" event={"ID":"71f3fe4f-d604-49ce-8364-6643e5c1afc1","Type":"ContainerStarted","Data":"96ec989d8696e7d2056fe75eab14f81fd5e9c9a91bfea4bcc75273974ec5f949"}
Sep 11 13:05:22.333875 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:22.333859    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-84qbs"
Sep 11 13:05:22.335081 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:22.335060    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f" event={"ID":"fefb898b-d767-4ab3-8b74-b8210565ffa4","Type":"ContainerStarted","Data":"83eaa654ed2d6d2b67c987fee655a8bbddffa087b7cec126a3e94baa65edaf72"}
Sep 11 13:05:22.335196 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:22.335136    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:22.352780 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:22.352732    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6" podStartSLOduration=2.415279861 podStartE2EDuration="5.35271315s" totalImagesPullingTime="2.937433289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:22.351585126 +0000 UTC m=+576.291286453" watchObservedRunningTime="2026-09-11 13:05:22.35271315 +0000 UTC m=+576.292414488"
Sep 11 13:05:22.378376 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:22.378322    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-84qbs" podStartSLOduration=2.5221616510000002 podStartE2EDuration="5.378306917s" totalImagesPullingTime="2.856145266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:22.377859827 +0000 UTC m=+576.317561160" watchObservedRunningTime="2026-09-11 13:05:22.378306917 +0000 UTC m=+576.318008249"
Sep 11 13:05:22.405249 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:22.405189    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f" podStartSLOduration=2.342223433 podStartE2EDuration="5.405175426s" totalImagesPullingTime="3.062951993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:22.403873523 +0000 UTC m=+576.343574857" watchObservedRunningTime="2026-09-11 13:05:22.405175426 +0000 UTC m=+576.344876759"
Sep 11 13:05:28.346323 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:28.346291    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:05:33.344178 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:33.344142    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jhml6"
Sep 11 13:05:33.344605 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:33.344510    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-84qbs"
Sep 11 13:05:36.331152 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.331123    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"]
Sep 11 13:05:36.335605 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.335587    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.339863 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.339837    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 11 13:05:36.340019 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.339890    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 13:05:36.340019 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.339840    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pjglm\""
Sep 11 13:05:36.340733 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.340714    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 13:05:36.348835 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.348816    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"]
Sep 11 13:05:36.440166 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.440141    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdr4m\" (UniqueName: \"kubernetes.io/projected/f9e446a0-924c-4a31-be2a-266d861fe6c3-kube-api-access-bdr4m\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.440294 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.440189    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e446a0-924c-4a31-be2a-266d861fe6c3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.440294 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.440217    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e446a0-924c-4a31-be2a-266d861fe6c3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.440294 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.440232    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e446a0-924c-4a31-be2a-266d861fe6c3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.440294 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.440258    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e446a0-924c-4a31-be2a-266d861fe6c3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.440294 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.440280    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e446a0-924c-4a31-be2a-266d861fe6c3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.541458 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.541415    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e446a0-924c-4a31-be2a-266d861fe6c3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.541588 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.541477    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e446a0-924c-4a31-be2a-266d861fe6c3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.541588 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.541499    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e446a0-924c-4a31-be2a-266d861fe6c3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.541588 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.541527    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e446a0-924c-4a31-be2a-266d861fe6c3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.541588 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.541558    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e446a0-924c-4a31-be2a-266d861fe6c3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.541816 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.541628    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdr4m\" (UniqueName: \"kubernetes.io/projected/f9e446a0-924c-4a31-be2a-266d861fe6c3-kube-api-access-bdr4m\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.541901 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.541883    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9e446a0-924c-4a31-be2a-266d861fe6c3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.541989 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.541965    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e446a0-924c-4a31-be2a-266d861fe6c3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.542049 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.541979    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9e446a0-924c-4a31-be2a-266d861fe6c3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.543602 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.543576    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9e446a0-924c-4a31-be2a-266d861fe6c3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.543945 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.543928    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9e446a0-924c-4a31-be2a-266d861fe6c3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.553076 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.553051    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdr4m\" (UniqueName: \"kubernetes.io/projected/f9e446a0-924c-4a31-be2a-266d861fe6c3-kube-api-access-bdr4m\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf\" (UID: \"f9e446a0-924c-4a31-be2a-266d861fe6c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.646247 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.646185    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:36.920453 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:36.920428    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"]
Sep 11 13:05:36.922478 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:05:36.922454    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e446a0_924c_4a31_be2a_266d861fe6c3.slice/crio-b63cf409c87b0b1a686b4e9877a51d09bf8d9973d8c804ae0e6ce88cfc342ed4 WatchSource:0}: Error finding container b63cf409c87b0b1a686b4e9877a51d09bf8d9973d8c804ae0e6ce88cfc342ed4: Status 404 returned error can't find the container with id b63cf409c87b0b1a686b4e9877a51d09bf8d9973d8c804ae0e6ce88cfc342ed4
Sep 11 13:05:37.417385 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:37.417290    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf" event={"ID":"f9e446a0-924c-4a31-be2a-266d861fe6c3","Type":"ContainerStarted","Data":"b63cf409c87b0b1a686b4e9877a51d09bf8d9973d8c804ae0e6ce88cfc342ed4"}
Sep 11 13:05:38.423974 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:38.423895    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf" event={"ID":"f9e446a0-924c-4a31-be2a-266d861fe6c3","Type":"ContainerStarted","Data":"067da629caf6efce6985b4181a7f251aa1a7f69747ef88659f5326d4cffdf5db"}
Sep 11 13:05:38.443642 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:38.443595    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf" podStartSLOduration=1.299279635 podStartE2EDuration="2.443580908s" totalImagesPullingTime="1.144301273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:38.441765926 +0000 UTC m=+592.381467259" watchObservedRunningTime="2026-09-11 13:05:38.443580908 +0000 UTC m=+592.383282242"
Sep 11 13:05:46.647009 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:46.646980    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:46.647351 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:46.647034    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:46.663883 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:46.663858    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:05:47.480109 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:05:47.480075    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf"
Sep 11 13:06:12.631992 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.631919    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"]
Sep 11 13:06:12.635463 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.635442    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.641040 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.641019    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 11 13:06:12.649469 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.649430    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"]
Sep 11 13:06:12.650995 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.650960    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.651111 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.651052    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc5lf\" (UniqueName: \"kubernetes.io/projected/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-kube-api-access-sc5lf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.651172 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.651136    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.651214 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.651175    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.651214 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.651202    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.651319 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.651267    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.752138 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.752110    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.752241 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.752166    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc5lf\" (UniqueName: \"kubernetes.io/projected/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-kube-api-access-sc5lf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.752241 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.752199    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.752241 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.752216    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.752241 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.752238    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.752450 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.752287    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.752699 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.752671    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.752808 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.752723    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.752868 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.752847    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.754648 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.754627    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.754756 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.754678    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.762600 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.762575    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc5lf\" (UniqueName: \"kubernetes.io/projected/18b2ad70-2bd1-483c-9c53-2e5c329b6bb3-kube-api-access-sc5lf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kr62x\" (UID: \"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:12.947113 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:12.947046    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:13.219358 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:13.219330    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"]
Sep 11 13:06:13.221407 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:06:13.221380    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18b2ad70_2bd1_483c_9c53_2e5c329b6bb3.slice/crio-dbedcf624dbddf737cfe3a787461524a8863f59dd02f6cf64de36fe9c3e4fd10 WatchSource:0}: Error finding container dbedcf624dbddf737cfe3a787461524a8863f59dd02f6cf64de36fe9c3e4fd10: Status 404 returned error can't find the container with id dbedcf624dbddf737cfe3a787461524a8863f59dd02f6cf64de36fe9c3e4fd10
Sep 11 13:06:13.222630 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:13.222612    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:06:13.598459 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:13.598403    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x" event={"ID":"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3","Type":"ContainerStarted","Data":"e6af1735e14ff5f16de7cf31208ccb5c4aa79d444892ccc8f61833d98c84dcc2"}
Sep 11 13:06:13.598604 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:13.598488    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x" event={"ID":"18b2ad70-2bd1-483c-9c53-2e5c329b6bb3","Type":"ContainerStarted","Data":"dbedcf624dbddf737cfe3a787461524a8863f59dd02f6cf64de36fe9c3e4fd10"}
Sep 11 13:06:13.618936 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:13.618890    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x" podStartSLOduration=1.435136542 podStartE2EDuration="1.618874389s" totalImagesPullingTime="183.737847ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:13.616413367 +0000 UTC m=+627.556114702" watchObservedRunningTime="2026-09-11 13:06:13.618874389 +0000 UTC m=+627.558575773"
Sep 11 13:06:22.947322 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:22.947276    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:22.947959 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:22.947356    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:22.963447 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:22.963408    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:06:23.658777 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:06:23.658750    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kr62x"
Sep 11 13:07:59.756667 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:07:59.756581    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-679f8bfbf-7v77p"]
Sep 11 13:07:59.760707 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:07:59.760670    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-679f8bfbf-7v77p"
Sep 11 13:07:59.764588 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:07:59.764568    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c8tfd\""
Sep 11 13:07:59.764698 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:07:59.764641    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 13:07:59.772595 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:07:59.772568    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-679f8bfbf-7v77p"]
Sep 11 13:07:59.927811 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:07:59.927778    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b613ea7b-3d4f-41cd-8c03-2d720452a603-tls-cert\") pod \"authorino-679f8bfbf-7v77p\" (UID: \"b613ea7b-3d4f-41cd-8c03-2d720452a603\") " pod="kuadrant-system/authorino-679f8bfbf-7v77p"
Sep 11 13:07:59.927984 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:07:59.927859    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2g88\" (UniqueName: \"kubernetes.io/projected/b613ea7b-3d4f-41cd-8c03-2d720452a603-kube-api-access-p2g88\") pod \"authorino-679f8bfbf-7v77p\" (UID: \"b613ea7b-3d4f-41cd-8c03-2d720452a603\") " pod="kuadrant-system/authorino-679f8bfbf-7v77p"
Sep 11 13:08:00.028862 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:00.028781    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b613ea7b-3d4f-41cd-8c03-2d720452a603-tls-cert\") pod \"authorino-679f8bfbf-7v77p\" (UID: \"b613ea7b-3d4f-41cd-8c03-2d720452a603\") " pod="kuadrant-system/authorino-679f8bfbf-7v77p"
Sep 11 13:08:00.028998 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:00.028890    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2g88\" (UniqueName: \"kubernetes.io/projected/b613ea7b-3d4f-41cd-8c03-2d720452a603-kube-api-access-p2g88\") pod \"authorino-679f8bfbf-7v77p\" (UID: \"b613ea7b-3d4f-41cd-8c03-2d720452a603\") " pod="kuadrant-system/authorino-679f8bfbf-7v77p"
Sep 11 13:08:00.031083 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:00.031052    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b613ea7b-3d4f-41cd-8c03-2d720452a603-tls-cert\") pod \"authorino-679f8bfbf-7v77p\" (UID: \"b613ea7b-3d4f-41cd-8c03-2d720452a603\") " pod="kuadrant-system/authorino-679f8bfbf-7v77p"
Sep 11 13:08:00.039027 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:00.039003    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2g88\" (UniqueName: \"kubernetes.io/projected/b613ea7b-3d4f-41cd-8c03-2d720452a603-kube-api-access-p2g88\") pod \"authorino-679f8bfbf-7v77p\" (UID: \"b613ea7b-3d4f-41cd-8c03-2d720452a603\") " pod="kuadrant-system/authorino-679f8bfbf-7v77p"
Sep 11 13:08:00.070652 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:00.070595    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-679f8bfbf-7v77p"
Sep 11 13:08:00.349006 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:00.348978    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-679f8bfbf-7v77p"]
Sep 11 13:08:00.351158 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:08:00.351135    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb613ea7b_3d4f_41cd_8c03_2d720452a603.slice/crio-133c8ed7fda3fab83d71349a3bedf9ac22483f54706644f8d865a0063356dd05 WatchSource:0}: Error finding container 133c8ed7fda3fab83d71349a3bedf9ac22483f54706644f8d865a0063356dd05: Status 404 returned error can't find the container with id 133c8ed7fda3fab83d71349a3bedf9ac22483f54706644f8d865a0063356dd05
Sep 11 13:08:01.145583 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:01.145545    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-679f8bfbf-7v77p" event={"ID":"b613ea7b-3d4f-41cd-8c03-2d720452a603","Type":"ContainerStarted","Data":"133c8ed7fda3fab83d71349a3bedf9ac22483f54706644f8d865a0063356dd05"}
Sep 11 13:08:03.159379 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:03.159344    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-679f8bfbf-7v77p" event={"ID":"b613ea7b-3d4f-41cd-8c03-2d720452a603","Type":"ContainerStarted","Data":"dccb5321e830f50354f4a9f175d2aef8c0b3a04204818789c4a3177d6d4261c6"}
Sep 11 13:08:03.180367 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:03.180302    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-679f8bfbf-7v77p" podStartSLOduration=1.5657200740000001 podStartE2EDuration="4.18028028s" totalImagesPullingTime="2.614560206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:08:03.17846651 +0000 UTC m=+737.118167846" watchObservedRunningTime="2026-09-11 13:08:03.18028028 +0000 UTC m=+737.119981614"
Sep 11 13:08:26.247929 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.247895    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"]
Sep 11 13:08:26.251707 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.251675    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:26.256207 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.256179    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-528ff6-serving-cert\""
Sep 11 13:08:26.268230 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.268198    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"]
Sep 11 13:08:26.357274 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.357242    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmrv7\" (UniqueName: \"kubernetes.io/projected/0ce32a64-e063-46a0-a11a-f94314f18528-kube-api-access-nmrv7\") pod \"maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:26.357274 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.357280    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ce32a64-e063-46a0-a11a-f94314f18528-metrics-certs\") pod \"maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:26.357535 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.357310    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ce32a64-e063-46a0-a11a-f94314f18528-maas-api-tls\") pod \"maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:26.357535 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.357377    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ce32a64-e063-46a0-a11a-f94314f18528-openshift-service-ca\") pod \"maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:26.458026 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.457996    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmrv7\" (UniqueName: \"kubernetes.io/projected/0ce32a64-e063-46a0-a11a-f94314f18528-kube-api-access-nmrv7\") pod \"maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:26.458026 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.458032    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ce32a64-e063-46a0-a11a-f94314f18528-metrics-certs\") pod \"maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:26.458258 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.458066    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ce32a64-e063-46a0-a11a-f94314f18528-maas-api-tls\") pod \"maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:26.458258 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.458084    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ce32a64-e063-46a0-a11a-f94314f18528-openshift-service-ca\") pod \"maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:26.458258 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:08:26.458231    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-serving-cert: secret "maas-api-e2e-inf-a-528ff6-serving-cert" not found
Sep 11 13:08:26.458465 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:08:26.458313    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ce32a64-e063-46a0-a11a-f94314f18528-maas-api-tls podName:0ce32a64-e063-46a0-a11a-f94314f18528 nodeName:}" failed. No retries permitted until 2026-09-11 13:08:26.958291238 +0000 UTC m=+760.897992564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0ce32a64-e063-46a0-a11a-f94314f18528-maas-api-tls") pod "maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b" (UID: "0ce32a64-e063-46a0-a11a-f94314f18528") : secret "maas-api-e2e-inf-a-528ff6-serving-cert" not found
Sep 11 13:08:26.458833 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.458810    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ce32a64-e063-46a0-a11a-f94314f18528-openshift-service-ca\") pod \"maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:26.460642 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.460618    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ce32a64-e063-46a0-a11a-f94314f18528-metrics-certs\") pod \"maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:26.468932 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.468912    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmrv7\" (UniqueName: \"kubernetes.io/projected/0ce32a64-e063-46a0-a11a-f94314f18528-kube-api-access-nmrv7\") pod \"maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:26.963221 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.963184    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ce32a64-e063-46a0-a11a-f94314f18528-maas-api-tls\") pod \"maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:26.965683 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:26.965657    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ce32a64-e063-46a0-a11a-f94314f18528-maas-api-tls\") pod \"maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:27.165578 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:27.165541    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:27.460480 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:27.460453    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"]
Sep 11 13:08:27.461219 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:08:27.461190    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ce32a64_e063_46a0_a11a_f94314f18528.slice/crio-4fa3cab930e585e14d21f981cb7bfe9b78a6635e90e5eca8cb46136022c388c9 WatchSource:0}: Error finding container 4fa3cab930e585e14d21f981cb7bfe9b78a6635e90e5eca8cb46136022c388c9: Status 404 returned error can't find the container with id 4fa3cab930e585e14d21f981cb7bfe9b78a6635e90e5eca8cb46136022c388c9
Sep 11 13:08:28.291042 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:28.291008    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b" event={"ID":"0ce32a64-e063-46a0-a11a-f94314f18528","Type":"ContainerStarted","Data":"396c278ad7d48dd40a01335fc9e829117eddc951815def2541cbd553f9d11aa5"}
Sep 11 13:08:28.291199 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:28.291058    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b" event={"ID":"0ce32a64-e063-46a0-a11a-f94314f18528","Type":"ContainerStarted","Data":"4fa3cab930e585e14d21f981cb7bfe9b78a6635e90e5eca8cb46136022c388c9"}
Sep 11 13:08:28.291199 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:28.291145    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:28.330192 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:28.330148    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b" podStartSLOduration=1.848107897 podStartE2EDuration="2.330136068s" totalImagesPullingTime="482.028171ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:08:28.329974443 +0000 UTC m=+762.269675777" watchObservedRunningTime="2026-09-11 13:08:28.330136068 +0000 UTC m=+762.269837400"
Sep 11 13:08:34.302200 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:34.302171    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:08:35.898228 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:35.898192    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"]
Sep 11 13:08:35.902098 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:35.902080    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:35.906433 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:35.906394    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-976b6f6f-serving-cert\""
Sep 11 13:08:35.927318 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:35.927292    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"]
Sep 11 13:08:36.048547 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:36.048509    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-metrics-certs\") pod \"maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:36.048729 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:36.048555    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgnhf\" (UniqueName: \"kubernetes.io/projected/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-kube-api-access-jgnhf\") pod \"maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:36.048729 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:36.048588    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-maas-api-tls\") pod \"maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:36.048729 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:36.048713    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-openshift-service-ca\") pod \"maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:36.150233 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:36.150157    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-openshift-service-ca\") pod \"maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:36.150233 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:36.150231    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-metrics-certs\") pod \"maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:36.150403 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:36.150257    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgnhf\" (UniqueName: \"kubernetes.io/projected/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-kube-api-access-jgnhf\") pod \"maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:36.150403 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:36.150292    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-maas-api-tls\") pod \"maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:36.150518 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:08:36.150412    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-serving-cert: secret "maas-api-e2e-ait-976b6f6f-serving-cert" not found
Sep 11 13:08:36.150518 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:08:36.150493    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-maas-api-tls podName:123c4a0d-b6d6-49b4-826b-c41ac8c00c3d nodeName:}" failed. No retries permitted until 2026-09-11 13:08:36.650473865 +0000 UTC m=+770.590175191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-maas-api-tls") pod "maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7" (UID: "123c4a0d-b6d6-49b4-826b-c41ac8c00c3d") : secret "maas-api-e2e-ait-976b6f6f-serving-cert" not found
Sep 11 13:08:36.150919 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:36.150896    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-openshift-service-ca\") pod \"maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:36.155926 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:36.155897    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-metrics-certs\") pod \"maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:36.161264 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:36.161233    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgnhf\" (UniqueName: \"kubernetes.io/projected/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-kube-api-access-jgnhf\") pod \"maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:36.655293 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:36.655253    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-maas-api-tls\") pod \"maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:36.658339 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:36.658307    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-maas-api-tls\") pod \"maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:36.813035 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:36.813009    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:37.121735 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:37.121707    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"]
Sep 11 13:08:37.123490 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:08:37.123453    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod123c4a0d_b6d6_49b4_826b_c41ac8c00c3d.slice/crio-5e4b5d393842a7abe63151c41958c11f7897cc8593724cf824ef30b3df8f96b2 WatchSource:0}: Error finding container 5e4b5d393842a7abe63151c41958c11f7897cc8593724cf824ef30b3df8f96b2: Status 404 returned error can't find the container with id 5e4b5d393842a7abe63151c41958c11f7897cc8593724cf824ef30b3df8f96b2
Sep 11 13:08:37.340852 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:37.340810    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7" event={"ID":"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d","Type":"ContainerStarted","Data":"5e4b5d393842a7abe63151c41958c11f7897cc8593724cf824ef30b3df8f96b2"}
Sep 11 13:08:38.347501 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:38.347465    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7" event={"ID":"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d","Type":"ContainerStarted","Data":"fbe5901b67fe3cb5c9f886cb8a595ba493778e864bf24dced449b0dd4956f1ca"}
Sep 11 13:08:38.347860 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:38.347531    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:39.440502 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.440449    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7" podStartSLOduration=3.965630595 podStartE2EDuration="4.440430263s" totalImagesPullingTime="474.799668ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:08:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:08:38.388085353 +0000 UTC m=+772.327786686" watchObservedRunningTime="2026-09-11 13:08:39.440430263 +0000 UTC m=+773.380131593"
Sep 11 13:08:39.442508 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.442487    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"]
Sep 11 13:08:39.446110 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.446094    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:39.448939 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.448918    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-efc023-serving-cert\""
Sep 11 13:08:39.462739 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.462717    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"]
Sep 11 13:08:39.585442 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.585398    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/615a2a25-0e85-48d1-9649-c7bbf0310993-metrics-certs\") pod \"maas-api-e2e-autores-efc023-5dddcd5b99-l42n8\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:39.585581 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.585469    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/615a2a25-0e85-48d1-9649-c7bbf0310993-maas-api-tls\") pod \"maas-api-e2e-autores-efc023-5dddcd5b99-l42n8\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:39.585581 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.585527    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlsqq\" (UniqueName: \"kubernetes.io/projected/615a2a25-0e85-48d1-9649-c7bbf0310993-kube-api-access-vlsqq\") pod \"maas-api-e2e-autores-efc023-5dddcd5b99-l42n8\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:39.585690 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.585600    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/615a2a25-0e85-48d1-9649-c7bbf0310993-openshift-service-ca\") pod \"maas-api-e2e-autores-efc023-5dddcd5b99-l42n8\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:39.686981 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.686954    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/615a2a25-0e85-48d1-9649-c7bbf0310993-metrics-certs\") pod \"maas-api-e2e-autores-efc023-5dddcd5b99-l42n8\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:39.687160 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.686995    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/615a2a25-0e85-48d1-9649-c7bbf0310993-maas-api-tls\") pod \"maas-api-e2e-autores-efc023-5dddcd5b99-l42n8\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:39.687160 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.687052    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlsqq\" (UniqueName: \"kubernetes.io/projected/615a2a25-0e85-48d1-9649-c7bbf0310993-kube-api-access-vlsqq\") pod \"maas-api-e2e-autores-efc023-5dddcd5b99-l42n8\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:39.687160 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.687081    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/615a2a25-0e85-48d1-9649-c7bbf0310993-openshift-service-ca\") pod \"maas-api-e2e-autores-efc023-5dddcd5b99-l42n8\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:39.687866 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.687839    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/615a2a25-0e85-48d1-9649-c7bbf0310993-openshift-service-ca\") pod \"maas-api-e2e-autores-efc023-5dddcd5b99-l42n8\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:39.689475 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.689457    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/615a2a25-0e85-48d1-9649-c7bbf0310993-maas-api-tls\") pod \"maas-api-e2e-autores-efc023-5dddcd5b99-l42n8\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:39.689654 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.689636    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/615a2a25-0e85-48d1-9649-c7bbf0310993-metrics-certs\") pod \"maas-api-e2e-autores-efc023-5dddcd5b99-l42n8\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:39.698592 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.698539    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlsqq\" (UniqueName: \"kubernetes.io/projected/615a2a25-0e85-48d1-9649-c7bbf0310993-kube-api-access-vlsqq\") pod \"maas-api-e2e-autores-efc023-5dddcd5b99-l42n8\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:39.757271 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:39.757243    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:40.031191 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:40.031166    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"]
Sep 11 13:08:40.033207 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:08:40.033174    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615a2a25_0e85_48d1_9649_c7bbf0310993.slice/crio-9cb953996319e89c1f4b3ccebd42ac04ff63474198d7b63ceef4cecd1eeb0a80 WatchSource:0}: Error finding container 9cb953996319e89c1f4b3ccebd42ac04ff63474198d7b63ceef4cecd1eeb0a80: Status 404 returned error can't find the container with id 9cb953996319e89c1f4b3ccebd42ac04ff63474198d7b63ceef4cecd1eeb0a80
Sep 11 13:08:40.361481 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:40.361385    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8" event={"ID":"615a2a25-0e85-48d1-9649-c7bbf0310993","Type":"ContainerStarted","Data":"9cb953996319e89c1f4b3ccebd42ac04ff63474198d7b63ceef4cecd1eeb0a80"}
Sep 11 13:08:41.368112 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:41.368078    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8" event={"ID":"615a2a25-0e85-48d1-9649-c7bbf0310993","Type":"ContainerStarted","Data":"b1f8f347ea3fd32d81dc122d643a8d4e11874b6dd4dcedf7b7202f2c68e7c285"}
Sep 11 13:08:41.368520 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:41.368233    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:41.408957 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:41.408910    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8" podStartSLOduration=2.0787818160000002 podStartE2EDuration="2.408895516s" totalImagesPullingTime="330.1137ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:08:41.406378549 +0000 UTC m=+775.346079886" watchObservedRunningTime="2026-09-11 13:08:41.408895516 +0000 UTC m=+775.348596847"
Sep 11 13:08:44.359678 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:44.359650    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:08:45.601411 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:45.601377    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"]
Sep 11 13:08:45.605346 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:45.605298    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:45.607920 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:45.607864    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a6ba91-serving-cert\""
Sep 11 13:08:45.628462 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:45.628435    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"]
Sep 11 13:08:45.749860 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:45.749820    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a391ffb7-78cd-44cb-b082-e20d1af2d410-metrics-certs\") pod \"maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:45.750045 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:45.749951    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a391ffb7-78cd-44cb-b082-e20d1af2d410-maas-api-tls\") pod \"maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:45.750045 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:45.749982    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a391ffb7-78cd-44cb-b082-e20d1af2d410-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:45.750045 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:45.750034    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqdw6\" (UniqueName: \"kubernetes.io/projected/a391ffb7-78cd-44cb-b082-e20d1af2d410-kube-api-access-sqdw6\") pod \"maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:45.851485 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:45.851380    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a391ffb7-78cd-44cb-b082-e20d1af2d410-metrics-certs\") pod \"maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:45.851651 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:45.851514    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a391ffb7-78cd-44cb-b082-e20d1af2d410-maas-api-tls\") pod \"maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:45.851651 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:45.851549    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a391ffb7-78cd-44cb-b082-e20d1af2d410-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:45.851651 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:45.851617    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqdw6\" (UniqueName: \"kubernetes.io/projected/a391ffb7-78cd-44cb-b082-e20d1af2d410-kube-api-access-sqdw6\") pod \"maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:45.851992 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:08:45.851964    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-serving-cert: secret "maas-api-e2e-inf-b-a6ba91-serving-cert" not found
Sep 11 13:08:45.852100 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:08:45.852038    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a391ffb7-78cd-44cb-b082-e20d1af2d410-maas-api-tls podName:a391ffb7-78cd-44cb-b082-e20d1af2d410 nodeName:}" failed. No retries permitted until 2026-09-11 13:08:46.352017794 +0000 UTC m=+780.291719122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a391ffb7-78cd-44cb-b082-e20d1af2d410-maas-api-tls") pod "maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg" (UID: "a391ffb7-78cd-44cb-b082-e20d1af2d410") : secret "maas-api-e2e-inf-b-a6ba91-serving-cert" not found
Sep 11 13:08:45.852794 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:45.852715    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a391ffb7-78cd-44cb-b082-e20d1af2d410-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:45.854735 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:45.854713    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a391ffb7-78cd-44cb-b082-e20d1af2d410-metrics-certs\") pod \"maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:45.878895 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:45.878868    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqdw6\" (UniqueName: \"kubernetes.io/projected/a391ffb7-78cd-44cb-b082-e20d1af2d410-kube-api-access-sqdw6\") pod \"maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:46.356969 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:46.356935    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a391ffb7-78cd-44cb-b082-e20d1af2d410-maas-api-tls\") pod \"maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:46.359450 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:46.359393    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a391ffb7-78cd-44cb-b082-e20d1af2d410-maas-api-tls\") pod \"maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:46.517820 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:46.517793    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:46.801214 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:46.801190    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"]
Sep 11 13:08:46.802817 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:08:46.802792    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda391ffb7_78cd_44cb_b082_e20d1af2d410.slice/crio-0d8f63f2acb0366105ab1aa0786ef009a2a6c03b89ac5798c9bc80f27e60eca3 WatchSource:0}: Error finding container 0d8f63f2acb0366105ab1aa0786ef009a2a6c03b89ac5798c9bc80f27e60eca3: Status 404 returned error can't find the container with id 0d8f63f2acb0366105ab1aa0786ef009a2a6c03b89ac5798c9bc80f27e60eca3
Sep 11 13:08:47.378505 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:47.378477    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:08:47.403890 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:47.403859    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg" event={"ID":"a391ffb7-78cd-44cb-b082-e20d1af2d410","Type":"ContainerStarted","Data":"692ab53576dcfb7ba7b5d1443b0fdbd07ed274b557e3b1ace2a91e69bd3c41c2"}
Sep 11 13:08:47.404007 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:47.403923    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:08:47.404007 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:47.403947    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg" event={"ID":"a391ffb7-78cd-44cb-b082-e20d1af2d410","Type":"ContainerStarted","Data":"0d8f63f2acb0366105ab1aa0786ef009a2a6c03b89ac5798c9bc80f27e60eca3"}
Sep 11 13:08:47.498987 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:47.498916    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg" podStartSLOduration=2.031761348 podStartE2EDuration="2.498885229s" totalImagesPullingTime="467.123881ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:08:47.498714749 +0000 UTC m=+781.438416100" watchObservedRunningTime="2026-09-11 13:08:47.498885229 +0000 UTC m=+781.438586563"
Sep 11 13:08:53.416384 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:08:53.416354    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:09:05.231576 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.231536    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"]
Sep 11 13:09:05.232031 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.231791    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7" podUID="123c4a0d-b6d6-49b4-826b-c41ac8c00c3d" containerName="maas-api" containerID="cri-o://fbe5901b67fe3cb5c9f886cb8a595ba493778e864bf24dced449b0dd4956f1ca" gracePeriod=30
Sep 11 13:09:05.498199 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.498146    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:09:05.506280 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.506256    2582 generic.go:356] "Generic (PLEG): container finished" podID="123c4a0d-b6d6-49b4-826b-c41ac8c00c3d" containerID="fbe5901b67fe3cb5c9f886cb8a595ba493778e864bf24dced449b0dd4956f1ca" exitCode=0
Sep 11 13:09:05.506406 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.506292    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7" event={"ID":"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d","Type":"ContainerDied","Data":"fbe5901b67fe3cb5c9f886cb8a595ba493778e864bf24dced449b0dd4956f1ca"}
Sep 11 13:09:05.506406 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.506295    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"
Sep 11 13:09:05.506406 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.506329    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7" event={"ID":"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d","Type":"ContainerDied","Data":"5e4b5d393842a7abe63151c41958c11f7897cc8593724cf824ef30b3df8f96b2"}
Sep 11 13:09:05.506406 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.506350    2582 scope.go:117] "RemoveContainer" containerID="fbe5901b67fe3cb5c9f886cb8a595ba493778e864bf24dced449b0dd4956f1ca"
Sep 11 13:09:05.515154 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.515135    2582 scope.go:117] "RemoveContainer" containerID="fbe5901b67fe3cb5c9f886cb8a595ba493778e864bf24dced449b0dd4956f1ca"
Sep 11 13:09:05.515436 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:09:05.515401    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbe5901b67fe3cb5c9f886cb8a595ba493778e864bf24dced449b0dd4956f1ca\": container with ID starting with fbe5901b67fe3cb5c9f886cb8a595ba493778e864bf24dced449b0dd4956f1ca not found: ID does not exist" containerID="fbe5901b67fe3cb5c9f886cb8a595ba493778e864bf24dced449b0dd4956f1ca"
Sep 11 13:09:05.515518 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.515450    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbe5901b67fe3cb5c9f886cb8a595ba493778e864bf24dced449b0dd4956f1ca"} err="failed to get container status \"fbe5901b67fe3cb5c9f886cb8a595ba493778e864bf24dced449b0dd4956f1ca\": rpc error: code = NotFound desc = could not find container \"fbe5901b67fe3cb5c9f886cb8a595ba493778e864bf24dced449b0dd4956f1ca\": container with ID starting with fbe5901b67fe3cb5c9f886cb8a595ba493778e864bf24dced449b0dd4956f1ca not found: ID does not exist"
Sep 11 13:09:05.644467 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.644434    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-maas-api-tls\") pod \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") "
Sep 11 13:09:05.644637 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.644548    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-openshift-service-ca\") pod \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") "
Sep 11 13:09:05.644637 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.644609    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-metrics-certs\") pod \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") "
Sep 11 13:09:05.644777 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.644667    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgnhf\" (UniqueName: \"kubernetes.io/projected/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-kube-api-access-jgnhf\") pod \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\" (UID: \"123c4a0d-b6d6-49b4-826b-c41ac8c00c3d\") "
Sep 11 13:09:05.644954 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.644928    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "123c4a0d-b6d6-49b4-826b-c41ac8c00c3d" (UID: "123c4a0d-b6d6-49b4-826b-c41ac8c00c3d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:09:05.645131 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.645113    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:09:05.646843 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.646814    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "123c4a0d-b6d6-49b4-826b-c41ac8c00c3d" (UID: "123c4a0d-b6d6-49b4-826b-c41ac8c00c3d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:09:05.646958 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.646937    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "123c4a0d-b6d6-49b4-826b-c41ac8c00c3d" (UID: "123c4a0d-b6d6-49b4-826b-c41ac8c00c3d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:09:05.647030 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.647003    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-kube-api-access-jgnhf" (OuterVolumeSpecName: "kube-api-access-jgnhf") pod "123c4a0d-b6d6-49b4-826b-c41ac8c00c3d" (UID: "123c4a0d-b6d6-49b4-826b-c41ac8c00c3d"). InnerVolumeSpecName "kube-api-access-jgnhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:09:05.745943 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.745912    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:09:05.745943 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.745941    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgnhf\" (UniqueName: \"kubernetes.io/projected/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-kube-api-access-jgnhf\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:09:05.746122 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.745952    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:09:05.837666 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.837640    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"]
Sep 11 13:09:05.846468 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:05.846438    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-976b6f6f-7ccddb7d5f-f7gf7"]
Sep 11 13:09:06.477555 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:06.477525    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="123c4a0d-b6d6-49b4-826b-c41ac8c00c3d" path="/var/lib/kubelet/pods/123c4a0d-b6d6-49b4-826b-c41ac8c00c3d/volumes"
Sep 11 13:09:16.081198 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.081118    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"]
Sep 11 13:09:16.081799 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.081778    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="123c4a0d-b6d6-49b4-826b-c41ac8c00c3d" containerName="maas-api"
Sep 11 13:09:16.081799 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.081799    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="123c4a0d-b6d6-49b4-826b-c41ac8c00c3d" containerName="maas-api"
Sep 11 13:09:16.081954 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.081938    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="123c4a0d-b6d6-49b4-826b-c41ac8c00c3d" containerName="maas-api"
Sep 11 13:09:16.085224 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.085203    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:16.088128 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.088109    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-87e2eb97-serving-cert\""
Sep 11 13:09:16.098505 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.098482    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"]
Sep 11 13:09:16.246291 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.246259    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74qqk\" (UniqueName: \"kubernetes.io/projected/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-kube-api-access-74qqk\") pod \"maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:16.246472 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.246301    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-openshift-service-ca\") pod \"maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:16.246472 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.246375    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-maas-api-tls\") pod \"maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:16.246472 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.246407    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-metrics-certs\") pod \"maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:16.347254 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.347178    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-maas-api-tls\") pod \"maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:16.347254 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.347217    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-metrics-certs\") pod \"maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:16.347490 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.347315    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74qqk\" (UniqueName: \"kubernetes.io/projected/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-kube-api-access-74qqk\") pod \"maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:16.347490 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.347353    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-openshift-service-ca\") pod \"maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:16.348015 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.347991    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-openshift-service-ca\") pod \"maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:16.349614 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.349585    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-metrics-certs\") pod \"maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:16.349750 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.349733    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-maas-api-tls\") pod \"maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:16.364894 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.364861    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74qqk\" (UniqueName: \"kubernetes.io/projected/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-kube-api-access-74qqk\") pod \"maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:16.398505 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.398479    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:16.681086 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:16.681059    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"]
Sep 11 13:09:16.682856 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:09:16.682832    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80cb9ac8_09a4_4dea_abee_36a29ba71f7a.slice/crio-8f838c7ada12ad178174101be4c39121fb9385ab75be8f48a9bcb546280cf6a6 WatchSource:0}: Error finding container 8f838c7ada12ad178174101be4c39121fb9385ab75be8f48a9bcb546280cf6a6: Status 404 returned error can't find the container with id 8f838c7ada12ad178174101be4c39121fb9385ab75be8f48a9bcb546280cf6a6
Sep 11 13:09:17.570628 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:17.570595    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l" event={"ID":"80cb9ac8-09a4-4dea-abee-36a29ba71f7a","Type":"ContainerStarted","Data":"49e931454c1b0b62561eee5a12501be10b708730e15a12e63a46ce6f4671b122"}
Sep 11 13:09:17.570995 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:17.570644    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l" event={"ID":"80cb9ac8-09a4-4dea-abee-36a29ba71f7a","Type":"ContainerStarted","Data":"8f838c7ada12ad178174101be4c39121fb9385ab75be8f48a9bcb546280cf6a6"}
Sep 11 13:09:17.570995 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:17.570664    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:17.666267 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:17.666219    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l" podStartSLOduration=1.28884799 podStartE2EDuration="1.666202214s" totalImagesPullingTime="377.354224ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:09:17.665226077 +0000 UTC m=+811.604927411" watchObservedRunningTime="2026-09-11 13:09:17.666202214 +0000 UTC m=+811.605903597"
Sep 11 13:09:23.582399 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:23.582373    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:52.154188 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.154146    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"]
Sep 11 13:09:52.154729 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.154414    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l" podUID="80cb9ac8-09a4-4dea-abee-36a29ba71f7a" containerName="maas-api" containerID="cri-o://49e931454c1b0b62561eee5a12501be10b708730e15a12e63a46ce6f4671b122" gracePeriod=30
Sep 11 13:09:52.428141 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.428112    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:52.493093 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.493059    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-metrics-certs\") pod \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") "
Sep 11 13:09:52.493298 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.493187    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-maas-api-tls\") pod \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") "
Sep 11 13:09:52.493298 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.493210    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74qqk\" (UniqueName: \"kubernetes.io/projected/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-kube-api-access-74qqk\") pod \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") "
Sep 11 13:09:52.493298 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.493270    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-openshift-service-ca\") pod \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\" (UID: \"80cb9ac8-09a4-4dea-abee-36a29ba71f7a\") "
Sep 11 13:09:52.493856 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.493819    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "80cb9ac8-09a4-4dea-abee-36a29ba71f7a" (UID: "80cb9ac8-09a4-4dea-abee-36a29ba71f7a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:09:52.495871 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.495824    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "80cb9ac8-09a4-4dea-abee-36a29ba71f7a" (UID: "80cb9ac8-09a4-4dea-abee-36a29ba71f7a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:09:52.495997 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.495850    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "80cb9ac8-09a4-4dea-abee-36a29ba71f7a" (UID: "80cb9ac8-09a4-4dea-abee-36a29ba71f7a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:09:52.495997 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.495849    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-kube-api-access-74qqk" (OuterVolumeSpecName: "kube-api-access-74qqk") pod "80cb9ac8-09a4-4dea-abee-36a29ba71f7a" (UID: "80cb9ac8-09a4-4dea-abee-36a29ba71f7a"). InnerVolumeSpecName "kube-api-access-74qqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:09:52.595144 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.595107    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:09:52.595144 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.595137    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74qqk\" (UniqueName: \"kubernetes.io/projected/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-kube-api-access-74qqk\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:09:52.595144 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.595152    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:09:52.595457 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.595162    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80cb9ac8-09a4-4dea-abee-36a29ba71f7a-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:09:52.769024 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.768984    2582 generic.go:356] "Generic (PLEG): container finished" podID="80cb9ac8-09a4-4dea-abee-36a29ba71f7a" containerID="49e931454c1b0b62561eee5a12501be10b708730e15a12e63a46ce6f4671b122" exitCode=0
Sep 11 13:09:52.769226 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.769043    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"
Sep 11 13:09:52.769226 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.769070    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l" event={"ID":"80cb9ac8-09a4-4dea-abee-36a29ba71f7a","Type":"ContainerDied","Data":"49e931454c1b0b62561eee5a12501be10b708730e15a12e63a46ce6f4671b122"}
Sep 11 13:09:52.769226 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.769126    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l" event={"ID":"80cb9ac8-09a4-4dea-abee-36a29ba71f7a","Type":"ContainerDied","Data":"8f838c7ada12ad178174101be4c39121fb9385ab75be8f48a9bcb546280cf6a6"}
Sep 11 13:09:52.769226 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.769148    2582 scope.go:117] "RemoveContainer" containerID="49e931454c1b0b62561eee5a12501be10b708730e15a12e63a46ce6f4671b122"
Sep 11 13:09:52.779982 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.779965    2582 scope.go:117] "RemoveContainer" containerID="49e931454c1b0b62561eee5a12501be10b708730e15a12e63a46ce6f4671b122"
Sep 11 13:09:52.780273 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:09:52.780252    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49e931454c1b0b62561eee5a12501be10b708730e15a12e63a46ce6f4671b122\": container with ID starting with 49e931454c1b0b62561eee5a12501be10b708730e15a12e63a46ce6f4671b122 not found: ID does not exist" containerID="49e931454c1b0b62561eee5a12501be10b708730e15a12e63a46ce6f4671b122"
Sep 11 13:09:52.780326 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.780279    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49e931454c1b0b62561eee5a12501be10b708730e15a12e63a46ce6f4671b122"} err="failed to get container status \"49e931454c1b0b62561eee5a12501be10b708730e15a12e63a46ce6f4671b122\": rpc error: code = NotFound desc = could not find container \"49e931454c1b0b62561eee5a12501be10b708730e15a12e63a46ce6f4671b122\": container with ID starting with 49e931454c1b0b62561eee5a12501be10b708730e15a12e63a46ce6f4671b122 not found: ID does not exist"
Sep 11 13:09:52.799746 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.799719    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"]
Sep 11 13:09:52.804930 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:52.804907    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-87e2eb97-7555d866bc-4qg8l"]
Sep 11 13:09:54.477158 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:54.477126    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80cb9ac8-09a4-4dea-abee-36a29ba71f7a" path="/var/lib/kubelet/pods/80cb9ac8-09a4-4dea-abee-36a29ba71f7a/volumes"
Sep 11 13:09:59.604328 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.604296    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"]
Sep 11 13:09:59.604813 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.604763    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80cb9ac8-09a4-4dea-abee-36a29ba71f7a" containerName="maas-api"
Sep 11 13:09:59.604813 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.604776    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="80cb9ac8-09a4-4dea-abee-36a29ba71f7a" containerName="maas-api"
Sep 11 13:09:59.604885 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.604847    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="80cb9ac8-09a4-4dea-abee-36a29ba71f7a" containerName="maas-api"
Sep 11 13:09:59.609444 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.609427    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:09:59.616007 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.615984    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-48769560-serving-cert\""
Sep 11 13:09:59.634547 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.634512    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"]
Sep 11 13:09:59.764255 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.764222    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10c21a0d-8d95-4d86-9a61-b560822895dc-metrics-certs\") pod \"maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:09:59.764455 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.764299    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10c21a0d-8d95-4d86-9a61-b560822895dc-maas-api-tls\") pod \"maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:09:59.764455 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.764335    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10c21a0d-8d95-4d86-9a61-b560822895dc-openshift-service-ca\") pod \"maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:09:59.764455 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.764353    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7zdj\" (UniqueName: \"kubernetes.io/projected/10c21a0d-8d95-4d86-9a61-b560822895dc-kube-api-access-g7zdj\") pod \"maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:09:59.865266 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.865165    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10c21a0d-8d95-4d86-9a61-b560822895dc-metrics-certs\") pod \"maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:09:59.865474 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.865344    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10c21a0d-8d95-4d86-9a61-b560822895dc-maas-api-tls\") pod \"maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:09:59.865474 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.865448    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10c21a0d-8d95-4d86-9a61-b560822895dc-openshift-service-ca\") pod \"maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:09:59.865609 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.865478    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7zdj\" (UniqueName: \"kubernetes.io/projected/10c21a0d-8d95-4d86-9a61-b560822895dc-kube-api-access-g7zdj\") pod \"maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:09:59.866237 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.866210    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10c21a0d-8d95-4d86-9a61-b560822895dc-openshift-service-ca\") pod \"maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:09:59.867847 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.867827    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10c21a0d-8d95-4d86-9a61-b560822895dc-metrics-certs\") pod \"maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:09:59.868110 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.868086    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10c21a0d-8d95-4d86-9a61-b560822895dc-maas-api-tls\") pod \"maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:09:59.885859 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.885833    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7zdj\" (UniqueName: \"kubernetes.io/projected/10c21a0d-8d95-4d86-9a61-b560822895dc-kube-api-access-g7zdj\") pod \"maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:09:59.922401 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:09:59.922364    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:10:00.241784 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:00.241755    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"]
Sep 11 13:10:00.244569 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:10:00.244542    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c21a0d_8d95_4d86_9a61_b560822895dc.slice/crio-e52cc03068079a6af33f0ebb263d8bdb0d0e61ed35f5a910f109ffe6b83e503b WatchSource:0}: Error finding container e52cc03068079a6af33f0ebb263d8bdb0d0e61ed35f5a910f109ffe6b83e503b: Status 404 returned error can't find the container with id e52cc03068079a6af33f0ebb263d8bdb0d0e61ed35f5a910f109ffe6b83e503b
Sep 11 13:10:00.813532 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:00.813493    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq" event={"ID":"10c21a0d-8d95-4d86-9a61-b560822895dc","Type":"ContainerStarted","Data":"8ecd106bbb9fe67d9dcc91a2724e0ec0db532b1b4bbceb5d1f8dfbf8e0df73e0"}
Sep 11 13:10:00.813997 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:00.813557    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq" event={"ID":"10c21a0d-8d95-4d86-9a61-b560822895dc","Type":"ContainerStarted","Data":"e52cc03068079a6af33f0ebb263d8bdb0d0e61ed35f5a910f109ffe6b83e503b"}
Sep 11 13:10:00.813997 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:00.813667    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:10:00.909401 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:00.909347    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq" podStartSLOduration=1.477531975 podStartE2EDuration="1.909331292s" totalImagesPullingTime="431.799317ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:09:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:10:00.907030002 +0000 UTC m=+854.846731335" watchObservedRunningTime="2026-09-11 13:10:00.909331292 +0000 UTC m=+854.849032625"
Sep 11 13:10:06.823954 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:06.823924    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:10:15.647893 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:15.647857    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"]
Sep 11 13:10:15.648296 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:15.648083    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8" podUID="615a2a25-0e85-48d1-9649-c7bbf0310993" containerName="maas-api" containerID="cri-o://b1f8f347ea3fd32d81dc122d643a8d4e11874b6dd4dcedf7b7202f2c68e7c285" gracePeriod=30
Sep 11 13:10:15.912344 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:15.912246    2582 generic.go:356] "Generic (PLEG): container finished" podID="615a2a25-0e85-48d1-9649-c7bbf0310993" containerID="b1f8f347ea3fd32d81dc122d643a8d4e11874b6dd4dcedf7b7202f2c68e7c285" exitCode=0
Sep 11 13:10:15.912549 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:15.912437    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8" event={"ID":"615a2a25-0e85-48d1-9649-c7bbf0310993","Type":"ContainerDied","Data":"b1f8f347ea3fd32d81dc122d643a8d4e11874b6dd4dcedf7b7202f2c68e7c285"}
Sep 11 13:10:15.912549 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:15.912528    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8" event={"ID":"615a2a25-0e85-48d1-9649-c7bbf0310993","Type":"ContainerDied","Data":"9cb953996319e89c1f4b3ccebd42ac04ff63474198d7b63ceef4cecd1eeb0a80"}
Sep 11 13:10:15.912796 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:15.912552    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9cb953996319e89c1f4b3ccebd42ac04ff63474198d7b63ceef4cecd1eeb0a80"
Sep 11 13:10:15.917560 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:15.917537    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:10:15.932555 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:15.932524    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/615a2a25-0e85-48d1-9649-c7bbf0310993-openshift-service-ca\") pod \"615a2a25-0e85-48d1-9649-c7bbf0310993\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") "
Sep 11 13:10:15.932688 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:15.932566    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/615a2a25-0e85-48d1-9649-c7bbf0310993-metrics-certs\") pod \"615a2a25-0e85-48d1-9649-c7bbf0310993\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") "
Sep 11 13:10:15.932688 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:15.932618    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlsqq\" (UniqueName: \"kubernetes.io/projected/615a2a25-0e85-48d1-9649-c7bbf0310993-kube-api-access-vlsqq\") pod \"615a2a25-0e85-48d1-9649-c7bbf0310993\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") "
Sep 11 13:10:15.932688 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:15.932655    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/615a2a25-0e85-48d1-9649-c7bbf0310993-maas-api-tls\") pod \"615a2a25-0e85-48d1-9649-c7bbf0310993\" (UID: \"615a2a25-0e85-48d1-9649-c7bbf0310993\") "
Sep 11 13:10:15.932955 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:15.932928    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/615a2a25-0e85-48d1-9649-c7bbf0310993-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "615a2a25-0e85-48d1-9649-c7bbf0310993" (UID: "615a2a25-0e85-48d1-9649-c7bbf0310993"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:10:15.934787 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:15.934754    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/615a2a25-0e85-48d1-9649-c7bbf0310993-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "615a2a25-0e85-48d1-9649-c7bbf0310993" (UID: "615a2a25-0e85-48d1-9649-c7bbf0310993"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:10:15.934787 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:15.934768    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/615a2a25-0e85-48d1-9649-c7bbf0310993-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "615a2a25-0e85-48d1-9649-c7bbf0310993" (UID: "615a2a25-0e85-48d1-9649-c7bbf0310993"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:10:15.934787 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:15.934758    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/615a2a25-0e85-48d1-9649-c7bbf0310993-kube-api-access-vlsqq" (OuterVolumeSpecName: "kube-api-access-vlsqq") pod "615a2a25-0e85-48d1-9649-c7bbf0310993" (UID: "615a2a25-0e85-48d1-9649-c7bbf0310993"). InnerVolumeSpecName "kube-api-access-vlsqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:10:16.033763 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:16.033725    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/615a2a25-0e85-48d1-9649-c7bbf0310993-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:10:16.033763 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:16.033758    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/615a2a25-0e85-48d1-9649-c7bbf0310993-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:10:16.033763 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:16.033769    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/615a2a25-0e85-48d1-9649-c7bbf0310993-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:10:16.034004 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:16.033777    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlsqq\" (UniqueName: \"kubernetes.io/projected/615a2a25-0e85-48d1-9649-c7bbf0310993-kube-api-access-vlsqq\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:10:16.917697 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:16.917602    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"
Sep 11 13:10:16.961475 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:16.961443    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"]
Sep 11 13:10:16.980824 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:16.980796    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-efc023-5dddcd5b99-l42n8"]
Sep 11 13:10:18.476686 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:18.476648    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="615a2a25-0e85-48d1-9649-c7bbf0310993" path="/var/lib/kubelet/pods/615a2a25-0e85-48d1-9649-c7bbf0310993/volumes"
Sep 11 13:10:22.873088 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:22.873049    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"]
Sep 11 13:10:22.873502 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:22.873310    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq" podUID="10c21a0d-8d95-4d86-9a61-b560822895dc" containerName="maas-api" containerID="cri-o://8ecd106bbb9fe67d9dcc91a2724e0ec0db532b1b4bbceb5d1f8dfbf8e0df73e0" gracePeriod=30
Sep 11 13:10:23.140052 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.139996    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:10:23.204010 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.203974    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10c21a0d-8d95-4d86-9a61-b560822895dc-openshift-service-ca\") pod \"10c21a0d-8d95-4d86-9a61-b560822895dc\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") "
Sep 11 13:10:23.204199 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.204069    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10c21a0d-8d95-4d86-9a61-b560822895dc-metrics-certs\") pod \"10c21a0d-8d95-4d86-9a61-b560822895dc\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") "
Sep 11 13:10:23.204199 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.204112    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10c21a0d-8d95-4d86-9a61-b560822895dc-maas-api-tls\") pod \"10c21a0d-8d95-4d86-9a61-b560822895dc\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") "
Sep 11 13:10:23.204199 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.204155    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7zdj\" (UniqueName: \"kubernetes.io/projected/10c21a0d-8d95-4d86-9a61-b560822895dc-kube-api-access-g7zdj\") pod \"10c21a0d-8d95-4d86-9a61-b560822895dc\" (UID: \"10c21a0d-8d95-4d86-9a61-b560822895dc\") "
Sep 11 13:10:23.204401 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.204375    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10c21a0d-8d95-4d86-9a61-b560822895dc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "10c21a0d-8d95-4d86-9a61-b560822895dc" (UID: "10c21a0d-8d95-4d86-9a61-b560822895dc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:10:23.204572 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.204554    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10c21a0d-8d95-4d86-9a61-b560822895dc-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:10:23.206354 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.206328    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10c21a0d-8d95-4d86-9a61-b560822895dc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "10c21a0d-8d95-4d86-9a61-b560822895dc" (UID: "10c21a0d-8d95-4d86-9a61-b560822895dc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:10:23.206488 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.206414    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10c21a0d-8d95-4d86-9a61-b560822895dc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "10c21a0d-8d95-4d86-9a61-b560822895dc" (UID: "10c21a0d-8d95-4d86-9a61-b560822895dc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:10:23.206561 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.206528    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10c21a0d-8d95-4d86-9a61-b560822895dc-kube-api-access-g7zdj" (OuterVolumeSpecName: "kube-api-access-g7zdj") pod "10c21a0d-8d95-4d86-9a61-b560822895dc" (UID: "10c21a0d-8d95-4d86-9a61-b560822895dc"). InnerVolumeSpecName "kube-api-access-g7zdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:10:23.306061 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.306026    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10c21a0d-8d95-4d86-9a61-b560822895dc-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:10:23.306211 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.306059    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7zdj\" (UniqueName: \"kubernetes.io/projected/10c21a0d-8d95-4d86-9a61-b560822895dc-kube-api-access-g7zdj\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:10:23.306211 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.306085    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10c21a0d-8d95-4d86-9a61-b560822895dc-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:10:23.958051 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.957965    2582 generic.go:356] "Generic (PLEG): container finished" podID="10c21a0d-8d95-4d86-9a61-b560822895dc" containerID="8ecd106bbb9fe67d9dcc91a2724e0ec0db532b1b4bbceb5d1f8dfbf8e0df73e0" exitCode=0
Sep 11 13:10:23.958051 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.958025    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"
Sep 11 13:10:23.958557 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.958048    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq" event={"ID":"10c21a0d-8d95-4d86-9a61-b560822895dc","Type":"ContainerDied","Data":"8ecd106bbb9fe67d9dcc91a2724e0ec0db532b1b4bbceb5d1f8dfbf8e0df73e0"}
Sep 11 13:10:23.958557 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.958108    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq" event={"ID":"10c21a0d-8d95-4d86-9a61-b560822895dc","Type":"ContainerDied","Data":"e52cc03068079a6af33f0ebb263d8bdb0d0e61ed35f5a910f109ffe6b83e503b"}
Sep 11 13:10:23.958557 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.958132    2582 scope.go:117] "RemoveContainer" containerID="8ecd106bbb9fe67d9dcc91a2724e0ec0db532b1b4bbceb5d1f8dfbf8e0df73e0"
Sep 11 13:10:23.968269 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.968249    2582 scope.go:117] "RemoveContainer" containerID="8ecd106bbb9fe67d9dcc91a2724e0ec0db532b1b4bbceb5d1f8dfbf8e0df73e0"
Sep 11 13:10:23.968582 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:10:23.968558    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ecd106bbb9fe67d9dcc91a2724e0ec0db532b1b4bbceb5d1f8dfbf8e0df73e0\": container with ID starting with 8ecd106bbb9fe67d9dcc91a2724e0ec0db532b1b4bbceb5d1f8dfbf8e0df73e0 not found: ID does not exist" containerID="8ecd106bbb9fe67d9dcc91a2724e0ec0db532b1b4bbceb5d1f8dfbf8e0df73e0"
Sep 11 13:10:23.968653 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:23.968591    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ecd106bbb9fe67d9dcc91a2724e0ec0db532b1b4bbceb5d1f8dfbf8e0df73e0"} err="failed to get container status \"8ecd106bbb9fe67d9dcc91a2724e0ec0db532b1b4bbceb5d1f8dfbf8e0df73e0\": rpc error: code = NotFound desc = could not find container \"8ecd106bbb9fe67d9dcc91a2724e0ec0db532b1b4bbceb5d1f8dfbf8e0df73e0\": container with ID starting with 8ecd106bbb9fe67d9dcc91a2724e0ec0db532b1b4bbceb5d1f8dfbf8e0df73e0 not found: ID does not exist"
Sep 11 13:10:24.005927 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:24.005893    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"]
Sep 11 13:10:24.014248 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:24.014226    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-48769560-7979b5c8c6-nj6nq"]
Sep 11 13:10:24.477515 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:24.477485    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10c21a0d-8d95-4d86-9a61-b560822895dc" path="/var/lib/kubelet/pods/10c21a0d-8d95-4d86-9a61-b560822895dc/volumes"
Sep 11 13:10:25.388634 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.388597    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"]
Sep 11 13:10:25.389139 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.389120    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10c21a0d-8d95-4d86-9a61-b560822895dc" containerName="maas-api"
Sep 11 13:10:25.389212 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.389144    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="10c21a0d-8d95-4d86-9a61-b560822895dc" containerName="maas-api"
Sep 11 13:10:25.389212 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.389173    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="615a2a25-0e85-48d1-9649-c7bbf0310993" containerName="maas-api"
Sep 11 13:10:25.389212 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.389179    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="615a2a25-0e85-48d1-9649-c7bbf0310993" containerName="maas-api"
Sep 11 13:10:25.389326 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.389254    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="10c21a0d-8d95-4d86-9a61-b560822895dc" containerName="maas-api"
Sep 11 13:10:25.389326 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.389266    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="615a2a25-0e85-48d1-9649-c7bbf0310993" containerName="maas-api"
Sep 11 13:10:25.393692 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.393673    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.396780 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.396726    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 13:10:25.396922 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.396903    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-a30ab725-openshift-default-dockercfg-jhc9j\""
Sep 11 13:10:25.408804 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.408782    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"]
Sep 11 13:10:25.528540 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.528504    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-token\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.528703 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.528575    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhg57\" (UniqueName: \"kubernetes.io/projected/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-kube-api-access-xhg57\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.528703 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.528642    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-data\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.528782 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.528703    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-podinfo\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.528782 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.528740    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-envoy\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.528782 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.528772    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-workload-certs\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.528875 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.528846    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-credential-socket\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.528954 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.528896    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-workload-socket\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.528954 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.528929    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istiod-ca-cert\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.629874 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.629827    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-envoy\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.630042 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.629887    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-workload-certs\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.630042 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.629944    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-credential-socket\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.630042 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.629972    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-workload-socket\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.630042 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.630002    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istiod-ca-cert\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.630042 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.630041    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-token\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.630332 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.630099    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhg57\" (UniqueName: \"kubernetes.io/projected/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-kube-api-access-xhg57\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.630332 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.630172    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-data\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.630332 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.630220    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-podinfo\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.630332 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.630311    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-workload-certs\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.630588 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.630367    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-credential-socket\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.630588 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.630512    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-workload-socket\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.630861 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.630838    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-data\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.631140 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.631120    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istiod-ca-cert\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.633111 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.633064    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-envoy\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.633668 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.633648    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-podinfo\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.646444 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.646377    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-token\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.646682 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.646662    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhg57\" (UniqueName: \"kubernetes.io/projected/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-kube-api-access-xhg57\") pod \"e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") " pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.704723 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.704686    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:25.992927 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:25.992880    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"]
Sep 11 13:10:26.977242 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:26.977197    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2" event={"ID":"647ccdc0-d78b-4c51-9d2d-d17d981e6bad","Type":"ContainerStarted","Data":"9dcaa0dffe07abfb59da3db58ad2bf9909865c44c8c3c5cccc928e0a8ef371f9"}
Sep 11 13:10:28.422670 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:28.422630    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 13:10:28.423037 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:28.422700    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 13:10:28.423037 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:28.422730    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 13:10:28.990191 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:28.990150    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2" event={"ID":"647ccdc0-d78b-4c51-9d2d-d17d981e6bad","Type":"ContainerStarted","Data":"12d67069b9928c50ba7644ade8e135a3ab35a7d55fadebec78c0790e9c45e112"}
Sep 11 13:10:29.026493 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:29.026445    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2" podStartSLOduration=1.600389241 podStartE2EDuration="4.026429105s" totalImagesPullingTime="2.426039864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:10:29.024624411 +0000 UTC m=+882.964325746" watchObservedRunningTime="2026-09-11 13:10:29.026429105 +0000 UTC m=+882.966130429"
Sep 11 13:10:29.705903 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:29.705871    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:29.711158 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:29.711133    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:29.995885 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:29.995854    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:29.996728 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:29.996708    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:33.376913 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:33.376880    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"]
Sep 11 13:10:33.380574 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:33.380550    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:33.383984 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:33.383959    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-a30ab725-serving-cert\""
Sep 11 13:10:33.406159 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:33.406129    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"]
Sep 11 13:10:33.505478 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:33.505450    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df191588-26f7-4edc-aeff-2e1bc15d9181-openshift-service-ca\") pod \"maas-api-e2e-derive-a30ab725-f979f7c85-tcs29\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:33.505598 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:33.505489    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mklhd\" (UniqueName: \"kubernetes.io/projected/df191588-26f7-4edc-aeff-2e1bc15d9181-kube-api-access-mklhd\") pod \"maas-api-e2e-derive-a30ab725-f979f7c85-tcs29\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:33.505598 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:33.505514    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df191588-26f7-4edc-aeff-2e1bc15d9181-maas-api-tls\") pod \"maas-api-e2e-derive-a30ab725-f979f7c85-tcs29\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:33.505687 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:33.505616    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df191588-26f7-4edc-aeff-2e1bc15d9181-metrics-certs\") pod \"maas-api-e2e-derive-a30ab725-f979f7c85-tcs29\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:33.607184 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:33.607146    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df191588-26f7-4edc-aeff-2e1bc15d9181-openshift-service-ca\") pod \"maas-api-e2e-derive-a30ab725-f979f7c85-tcs29\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:33.607184 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:33.607187    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mklhd\" (UniqueName: \"kubernetes.io/projected/df191588-26f7-4edc-aeff-2e1bc15d9181-kube-api-access-mklhd\") pod \"maas-api-e2e-derive-a30ab725-f979f7c85-tcs29\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:33.607461 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:33.607214    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df191588-26f7-4edc-aeff-2e1bc15d9181-maas-api-tls\") pod \"maas-api-e2e-derive-a30ab725-f979f7c85-tcs29\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:33.607461 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:10:33.607318    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-serving-cert: secret "maas-api-e2e-derive-a30ab725-serving-cert" not found
Sep 11 13:10:33.607461 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:10:33.607382    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df191588-26f7-4edc-aeff-2e1bc15d9181-maas-api-tls podName:df191588-26f7-4edc-aeff-2e1bc15d9181 nodeName:}" failed. No retries permitted until 2026-09-11 13:10:34.107362667 +0000 UTC m=+888.047063983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/df191588-26f7-4edc-aeff-2e1bc15d9181-maas-api-tls") pod "maas-api-e2e-derive-a30ab725-f979f7c85-tcs29" (UID: "df191588-26f7-4edc-aeff-2e1bc15d9181") : secret "maas-api-e2e-derive-a30ab725-serving-cert" not found
Sep 11 13:10:33.607629 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:33.607497    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df191588-26f7-4edc-aeff-2e1bc15d9181-metrics-certs\") pod \"maas-api-e2e-derive-a30ab725-f979f7c85-tcs29\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:33.608058 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:33.607992    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df191588-26f7-4edc-aeff-2e1bc15d9181-openshift-service-ca\") pod \"maas-api-e2e-derive-a30ab725-f979f7c85-tcs29\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:33.610035 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:33.610012    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df191588-26f7-4edc-aeff-2e1bc15d9181-metrics-certs\") pod \"maas-api-e2e-derive-a30ab725-f979f7c85-tcs29\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:33.633180 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:33.633070    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mklhd\" (UniqueName: \"kubernetes.io/projected/df191588-26f7-4edc-aeff-2e1bc15d9181-kube-api-access-mklhd\") pod \"maas-api-e2e-derive-a30ab725-f979f7c85-tcs29\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:34.113603 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:34.113566    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df191588-26f7-4edc-aeff-2e1bc15d9181-maas-api-tls\") pod \"maas-api-e2e-derive-a30ab725-f979f7c85-tcs29\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:34.116023 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:34.115998    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df191588-26f7-4edc-aeff-2e1bc15d9181-maas-api-tls\") pod \"maas-api-e2e-derive-a30ab725-f979f7c85-tcs29\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:34.329029 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:34.328998    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:34.638673 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:34.638645    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"]
Sep 11 13:10:34.640398 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:10:34.640363    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf191588_26f7_4edc_aeff_2e1bc15d9181.slice/crio-bc0208bc88e20a47316ce65f916b814fcfeb9d2c6c37082afe2a064c8d706782 WatchSource:0}: Error finding container bc0208bc88e20a47316ce65f916b814fcfeb9d2c6c37082afe2a064c8d706782: Status 404 returned error can't find the container with id bc0208bc88e20a47316ce65f916b814fcfeb9d2c6c37082afe2a064c8d706782
Sep 11 13:10:35.026196 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:35.026159    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29" event={"ID":"df191588-26f7-4edc-aeff-2e1bc15d9181","Type":"ContainerStarted","Data":"bc0208bc88e20a47316ce65f916b814fcfeb9d2c6c37082afe2a064c8d706782"}
Sep 11 13:10:36.032976 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:36.032933    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29" event={"ID":"df191588-26f7-4edc-aeff-2e1bc15d9181","Type":"ContainerStarted","Data":"c191c41e50275f3cdd999bda979c5e8e528dceebb2245bc6037fa12fad64f598"}
Sep 11 13:10:36.033386 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:36.032995    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:36.078269 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:36.078227    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29" podStartSLOduration=2.581883446 podStartE2EDuration="3.078211376s" totalImagesPullingTime="496.32793ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:10:36.078196558 +0000 UTC m=+890.017897891" watchObservedRunningTime="2026-09-11 13:10:36.078211376 +0000 UTC m=+890.017912709"
Sep 11 13:10:42.042584 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:42.042510    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:10:54.138063 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:54.138027    2582 generic.go:356] "Generic (PLEG): container finished" podID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" containerID="12d67069b9928c50ba7644ade8e135a3ab35a7d55fadebec78c0790e9c45e112" exitCode=0
Sep 11 13:10:54.138615 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:54.138106    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2" event={"ID":"647ccdc0-d78b-4c51-9d2d-d17d981e6bad","Type":"ContainerDied","Data":"12d67069b9928c50ba7644ade8e135a3ab35a7d55fadebec78c0790e9c45e112"}
Sep 11 13:10:54.138742 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:54.138725    2582 scope.go:117] "RemoveContainer" containerID="12d67069b9928c50ba7644ade8e135a3ab35a7d55fadebec78c0790e9c45e112"
Sep 11 13:10:54.139862 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:54.139835    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 13:10:54.139929 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:54.139903    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 13:10:54.139965 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:54.139935    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 13:10:55.144823 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:55.144788    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2" event={"ID":"647ccdc0-d78b-4c51-9d2d-d17d981e6bad","Type":"ContainerStarted","Data":"64104b7025840687c53812150624c5e000a388ecde16c797471efd944caf7e0f"}
Sep 11 13:10:55.704892 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:55.704850    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:10:55.705989 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:55.705957    2582 patch_prober.go:28] interesting pod/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 13:10:55.706109 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:55.706016    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2" podUID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 13:10:56.705800 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:56.705763    2582 patch_prober.go:28] interesting pod/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 13:10:56.706245 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:56.705823    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2" podUID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 13:10:57.705588 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:57.705556    2582 patch_prober.go:28] interesting pod/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 13:10:57.705765 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:57.705612    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2" podUID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 13:10:58.705439 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:58.705382    2582 patch_prober.go:28] interesting pod/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 13:10:58.705815 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:58.705464    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2" podUID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 13:10:59.705655 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:59.705615    2582 patch_prober.go:28] interesting pod/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 13:10:59.706013 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:59.705683    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2" podUID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 13:10:59.996439 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:10:59.996395    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:11:00.709569 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:00.709538    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:11:00.710591 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:00.710570    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:11:09.122348 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.122313    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"]
Sep 11 13:11:09.129614 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.129592    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:09.135123 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.135101    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-ecc2b9-serving-cert\""
Sep 11 13:11:09.146651 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.146610    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"]
Sep 11 13:11:09.276319 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.276280    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnkjv\" (UniqueName: \"kubernetes.io/projected/746d6a16-775e-43ba-b4d7-42de068b6dbe-kube-api-access-xnkjv\") pod \"maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:09.276503 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.276356    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746d6a16-775e-43ba-b4d7-42de068b6dbe-metrics-certs\") pod \"maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:09.276503 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.276377    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/746d6a16-775e-43ba-b4d7-42de068b6dbe-maas-api-tls\") pod \"maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:09.276503 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.276413    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/746d6a16-775e-43ba-b4d7-42de068b6dbe-openshift-service-ca\") pod \"maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:09.377306 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.377224    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/746d6a16-775e-43ba-b4d7-42de068b6dbe-openshift-service-ca\") pod \"maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:09.377306 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.377295    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnkjv\" (UniqueName: \"kubernetes.io/projected/746d6a16-775e-43ba-b4d7-42de068b6dbe-kube-api-access-xnkjv\") pod \"maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:09.377520 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.377350    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746d6a16-775e-43ba-b4d7-42de068b6dbe-metrics-certs\") pod \"maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:09.377520 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.377396    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/746d6a16-775e-43ba-b4d7-42de068b6dbe-maas-api-tls\") pod \"maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:09.377919 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.377897    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/746d6a16-775e-43ba-b4d7-42de068b6dbe-openshift-service-ca\") pod \"maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:09.379819 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.379794    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746d6a16-775e-43ba-b4d7-42de068b6dbe-metrics-certs\") pod \"maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:09.379980 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.379958    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/746d6a16-775e-43ba-b4d7-42de068b6dbe-maas-api-tls\") pod \"maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:09.387848 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.387826    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnkjv\" (UniqueName: \"kubernetes.io/projected/746d6a16-775e-43ba-b4d7-42de068b6dbe-kube-api-access-xnkjv\") pod \"maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:09.440571 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.440548    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:09.724702 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:09.724678    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"]
Sep 11 13:11:09.725937 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:11:09.725906    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod746d6a16_775e_43ba_b4d7_42de068b6dbe.slice/crio-1b3e82530e5c6de2b5b5a52067e77f497337f32880ad630ebe82b93c3ba3b55d WatchSource:0}: Error finding container 1b3e82530e5c6de2b5b5a52067e77f497337f32880ad630ebe82b93c3ba3b55d: Status 404 returned error can't find the container with id 1b3e82530e5c6de2b5b5a52067e77f497337f32880ad630ebe82b93c3ba3b55d
Sep 11 13:11:10.249743 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:10.249709    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b" event={"ID":"746d6a16-775e-43ba-b4d7-42de068b6dbe","Type":"ContainerStarted","Data":"481d20c916cb73543feb4b35bba24c664ce0687fa41cf897a5d0378cfb236c29"}
Sep 11 13:11:10.250105 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:10.249776    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b" event={"ID":"746d6a16-775e-43ba-b4d7-42de068b6dbe","Type":"ContainerStarted","Data":"1b3e82530e5c6de2b5b5a52067e77f497337f32880ad630ebe82b93c3ba3b55d"}
Sep 11 13:11:10.250105 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:10.249893    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:10.286154 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:10.286101    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b" podStartSLOduration=0.883500081 podStartE2EDuration="1.286084601s" totalImagesPullingTime="402.58452ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:11:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:11:10.285032586 +0000 UTC m=+924.224733931" watchObservedRunningTime="2026-09-11 13:11:10.286084601 +0000 UTC m=+924.225785934"
Sep 11 13:11:12.502301 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:12.502264    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"]
Sep 11 13:11:12.502713 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:12.502486    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29" podUID="df191588-26f7-4edc-aeff-2e1bc15d9181" containerName="maas-api" containerID="cri-o://c191c41e50275f3cdd999bda979c5e8e528dceebb2245bc6037fa12fad64f598" gracePeriod=30
Sep 11 13:11:12.754378 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:12.754316    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:11:12.812861 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:12.812411    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df191588-26f7-4edc-aeff-2e1bc15d9181-openshift-service-ca\") pod \"df191588-26f7-4edc-aeff-2e1bc15d9181\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") "
Sep 11 13:11:12.812861 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:12.812486    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mklhd\" (UniqueName: \"kubernetes.io/projected/df191588-26f7-4edc-aeff-2e1bc15d9181-kube-api-access-mklhd\") pod \"df191588-26f7-4edc-aeff-2e1bc15d9181\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") "
Sep 11 13:11:12.812861 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:12.812541    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df191588-26f7-4edc-aeff-2e1bc15d9181-metrics-certs\") pod \"df191588-26f7-4edc-aeff-2e1bc15d9181\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") "
Sep 11 13:11:12.812861 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:12.812636    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df191588-26f7-4edc-aeff-2e1bc15d9181-maas-api-tls\") pod \"df191588-26f7-4edc-aeff-2e1bc15d9181\" (UID: \"df191588-26f7-4edc-aeff-2e1bc15d9181\") "
Sep 11 13:11:12.812861 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:12.812824    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df191588-26f7-4edc-aeff-2e1bc15d9181-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "df191588-26f7-4edc-aeff-2e1bc15d9181" (UID: "df191588-26f7-4edc-aeff-2e1bc15d9181"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:11:12.813234 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:12.813034    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df191588-26f7-4edc-aeff-2e1bc15d9181-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:12.815310 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:12.815280    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df191588-26f7-4edc-aeff-2e1bc15d9181-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "df191588-26f7-4edc-aeff-2e1bc15d9181" (UID: "df191588-26f7-4edc-aeff-2e1bc15d9181"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:11:12.815310 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:12.815293    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df191588-26f7-4edc-aeff-2e1bc15d9181-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "df191588-26f7-4edc-aeff-2e1bc15d9181" (UID: "df191588-26f7-4edc-aeff-2e1bc15d9181"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:11:12.815489 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:12.815296    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df191588-26f7-4edc-aeff-2e1bc15d9181-kube-api-access-mklhd" (OuterVolumeSpecName: "kube-api-access-mklhd") pod "df191588-26f7-4edc-aeff-2e1bc15d9181" (UID: "df191588-26f7-4edc-aeff-2e1bc15d9181"). InnerVolumeSpecName "kube-api-access-mklhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:11:12.914463 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:12.914397    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df191588-26f7-4edc-aeff-2e1bc15d9181-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:12.914463 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:12.914468    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df191588-26f7-4edc-aeff-2e1bc15d9181-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:12.914663 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:12.914483    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mklhd\" (UniqueName: \"kubernetes.io/projected/df191588-26f7-4edc-aeff-2e1bc15d9181-kube-api-access-mklhd\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:13.022193 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:13.022093    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"]
Sep 11 13:11:13.022393 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:13.022364    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2" podUID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" containerName="istio-proxy" containerID="cri-o://64104b7025840687c53812150624c5e000a388ecde16c797471efd944caf7e0f" gracePeriod=30
Sep 11 13:11:13.268974 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:13.268937    2582 generic.go:356] "Generic (PLEG): container finished" podID="df191588-26f7-4edc-aeff-2e1bc15d9181" containerID="c191c41e50275f3cdd999bda979c5e8e528dceebb2245bc6037fa12fad64f598" exitCode=0
Sep 11 13:11:13.268974 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:13.268978    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29" event={"ID":"df191588-26f7-4edc-aeff-2e1bc15d9181","Type":"ContainerDied","Data":"c191c41e50275f3cdd999bda979c5e8e528dceebb2245bc6037fa12fad64f598"}
Sep 11 13:11:13.269169 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:13.268982    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"
Sep 11 13:11:13.269169 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:13.269025    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29" event={"ID":"df191588-26f7-4edc-aeff-2e1bc15d9181","Type":"ContainerDied","Data":"bc0208bc88e20a47316ce65f916b814fcfeb9d2c6c37082afe2a064c8d706782"}
Sep 11 13:11:13.269169 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:13.269048    2582 scope.go:117] "RemoveContainer" containerID="c191c41e50275f3cdd999bda979c5e8e528dceebb2245bc6037fa12fad64f598"
Sep 11 13:11:13.281077 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:13.281015    2582 scope.go:117] "RemoveContainer" containerID="c191c41e50275f3cdd999bda979c5e8e528dceebb2245bc6037fa12fad64f598"
Sep 11 13:11:13.281572 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:11:13.281539    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c191c41e50275f3cdd999bda979c5e8e528dceebb2245bc6037fa12fad64f598\": container with ID starting with c191c41e50275f3cdd999bda979c5e8e528dceebb2245bc6037fa12fad64f598 not found: ID does not exist" containerID="c191c41e50275f3cdd999bda979c5e8e528dceebb2245bc6037fa12fad64f598"
Sep 11 13:11:13.281709 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:13.281581    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c191c41e50275f3cdd999bda979c5e8e528dceebb2245bc6037fa12fad64f598"} err="failed to get container status \"c191c41e50275f3cdd999bda979c5e8e528dceebb2245bc6037fa12fad64f598\": rpc error: code = NotFound desc = could not find container \"c191c41e50275f3cdd999bda979c5e8e528dceebb2245bc6037fa12fad64f598\": container with ID starting with c191c41e50275f3cdd999bda979c5e8e528dceebb2245bc6037fa12fad64f598 not found: ID does not exist"
Sep 11 13:11:13.303657 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:13.303628    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"]
Sep 11 13:11:13.311536 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:13.311507    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a30ab725-f979f7c85-tcs29"]
Sep 11 13:11:14.477031 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:14.476990    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df191588-26f7-4edc-aeff-2e1bc15d9181" path="/var/lib/kubelet/pods/df191588-26f7-4edc-aeff-2e1bc15d9181/volumes"
Sep 11 13:11:15.712463 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:15.712409    2582 patch_prober.go:28] interesting pod/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 13:11:15.712826 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:15.712485    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2" podUID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:11:16.261707 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:16.261672    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:18.283948 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.283880    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:11:18.299793 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.299766    2582 generic.go:356] "Generic (PLEG): container finished" podID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" containerID="64104b7025840687c53812150624c5e000a388ecde16c797471efd944caf7e0f" exitCode=0
Sep 11 13:11:18.299941 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.299817    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"
Sep 11 13:11:18.299941 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.299855    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2" event={"ID":"647ccdc0-d78b-4c51-9d2d-d17d981e6bad","Type":"ContainerDied","Data":"64104b7025840687c53812150624c5e000a388ecde16c797471efd944caf7e0f"}
Sep 11 13:11:18.299941 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.299906    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2" event={"ID":"647ccdc0-d78b-4c51-9d2d-d17d981e6bad","Type":"ContainerDied","Data":"9dcaa0dffe07abfb59da3db58ad2bf9909865c44c8c3c5cccc928e0a8ef371f9"}
Sep 11 13:11:18.299941 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.299931    2582 scope.go:117] "RemoveContainer" containerID="64104b7025840687c53812150624c5e000a388ecde16c797471efd944caf7e0f"
Sep 11 13:11:18.308743 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.308726    2582 scope.go:117] "RemoveContainer" containerID="12d67069b9928c50ba7644ade8e135a3ab35a7d55fadebec78c0790e9c45e112"
Sep 11 13:11:18.316651 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.316631    2582 scope.go:117] "RemoveContainer" containerID="64104b7025840687c53812150624c5e000a388ecde16c797471efd944caf7e0f"
Sep 11 13:11:18.316927 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:11:18.316895    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64104b7025840687c53812150624c5e000a388ecde16c797471efd944caf7e0f\": container with ID starting with 64104b7025840687c53812150624c5e000a388ecde16c797471efd944caf7e0f not found: ID does not exist" containerID="64104b7025840687c53812150624c5e000a388ecde16c797471efd944caf7e0f"
Sep 11 13:11:18.316996 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.316941    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64104b7025840687c53812150624c5e000a388ecde16c797471efd944caf7e0f"} err="failed to get container status \"64104b7025840687c53812150624c5e000a388ecde16c797471efd944caf7e0f\": rpc error: code = NotFound desc = could not find container \"64104b7025840687c53812150624c5e000a388ecde16c797471efd944caf7e0f\": container with ID starting with 64104b7025840687c53812150624c5e000a388ecde16c797471efd944caf7e0f not found: ID does not exist"
Sep 11 13:11:18.316996 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.316965    2582 scope.go:117] "RemoveContainer" containerID="12d67069b9928c50ba7644ade8e135a3ab35a7d55fadebec78c0790e9c45e112"
Sep 11 13:11:18.317236 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:11:18.317219    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12d67069b9928c50ba7644ade8e135a3ab35a7d55fadebec78c0790e9c45e112\": container with ID starting with 12d67069b9928c50ba7644ade8e135a3ab35a7d55fadebec78c0790e9c45e112 not found: ID does not exist" containerID="12d67069b9928c50ba7644ade8e135a3ab35a7d55fadebec78c0790e9c45e112"
Sep 11 13:11:18.317276 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.317243    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12d67069b9928c50ba7644ade8e135a3ab35a7d55fadebec78c0790e9c45e112"} err="failed to get container status \"12d67069b9928c50ba7644ade8e135a3ab35a7d55fadebec78c0790e9c45e112\": rpc error: code = NotFound desc = could not find container \"12d67069b9928c50ba7644ade8e135a3ab35a7d55fadebec78c0790e9c45e112\": container with ID starting with 12d67069b9928c50ba7644ade8e135a3ab35a7d55fadebec78c0790e9c45e112 not found: ID does not exist"
Sep 11 13:11:18.368309 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.368279    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-podinfo\") pod \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") "
Sep 11 13:11:18.368475 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.368314    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-token\") pod \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") "
Sep 11 13:11:18.368475 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.368372    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-envoy\") pod \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") "
Sep 11 13:11:18.368475 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.368409    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-credential-socket\") pod \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") "
Sep 11 13:11:18.368475 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.368456    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhg57\" (UniqueName: \"kubernetes.io/projected/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-kube-api-access-xhg57\") pod \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") "
Sep 11 13:11:18.368698 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.368534    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-workload-socket\") pod \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") "
Sep 11 13:11:18.368698 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.368561    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istiod-ca-cert\") pod \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") "
Sep 11 13:11:18.368698 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.368592    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-data\") pod \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") "
Sep 11 13:11:18.368698 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.368610    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-workload-certs\") pod \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\" (UID: \"647ccdc0-d78b-4c51-9d2d-d17d981e6bad\") "
Sep 11 13:11:18.368898 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.368850    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "647ccdc0-d78b-4c51-9d2d-d17d981e6bad" (UID: "647ccdc0-d78b-4c51-9d2d-d17d981e6bad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:11:18.368980 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.368949    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "647ccdc0-d78b-4c51-9d2d-d17d981e6bad" (UID: "647ccdc0-d78b-4c51-9d2d-d17d981e6bad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:11:18.368980 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.368961    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-workload-socket\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:18.369084 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.369067    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "647ccdc0-d78b-4c51-9d2d-d17d981e6bad" (UID: "647ccdc0-d78b-4c51-9d2d-d17d981e6bad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:11:18.369173 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.369147    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-data" (OuterVolumeSpecName: "istio-data") pod "647ccdc0-d78b-4c51-9d2d-d17d981e6bad" (UID: "647ccdc0-d78b-4c51-9d2d-d17d981e6bad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:11:18.369298 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.369185    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "647ccdc0-d78b-4c51-9d2d-d17d981e6bad" (UID: "647ccdc0-d78b-4c51-9d2d-d17d981e6bad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:11:18.371122 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.371091    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "647ccdc0-d78b-4c51-9d2d-d17d981e6bad" (UID: "647ccdc0-d78b-4c51-9d2d-d17d981e6bad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:11:18.371229 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.371154    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-kube-api-access-xhg57" (OuterVolumeSpecName: "kube-api-access-xhg57") pod "647ccdc0-d78b-4c51-9d2d-d17d981e6bad" (UID: "647ccdc0-d78b-4c51-9d2d-d17d981e6bad"). InnerVolumeSpecName "kube-api-access-xhg57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:11:18.371229 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.371191    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "647ccdc0-d78b-4c51-9d2d-d17d981e6bad" (UID: "647ccdc0-d78b-4c51-9d2d-d17d981e6bad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:11:18.371338 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.371226    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-token" (OuterVolumeSpecName: "istio-token") pod "647ccdc0-d78b-4c51-9d2d-d17d981e6bad" (UID: "647ccdc0-d78b-4c51-9d2d-d17d981e6bad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:11:18.469914 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.469887    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhg57\" (UniqueName: \"kubernetes.io/projected/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-kube-api-access-xhg57\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:18.469914 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.469912    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istiod-ca-cert\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:18.469914 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.469922    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-data\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:18.470140 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.469932    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-workload-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:18.470140 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.470042    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-podinfo\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:18.470140 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.470068    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-token\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:18.470140 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.470081    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-istio-envoy\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:18.470140 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.470095    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/647ccdc0-d78b-4c51-9d2d-d17d981e6bad-credential-socket\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:18.618643 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.618563    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"]
Sep 11 13:11:18.623572 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:18.623546    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-a30ab725-openshift-default-b4549fdb4-xfwz2"]
Sep 11 13:11:20.476811 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:20.476776    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" path="/var/lib/kubelet/pods/647ccdc0-d78b-4c51-9d2d-d17d981e6bad/volumes"
Sep 11 13:11:47.524690 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:47.524644    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"]
Sep 11 13:11:47.525120 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:47.524835    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b" podUID="746d6a16-775e-43ba-b4d7-42de068b6dbe" containerName="maas-api" containerID="cri-o://481d20c916cb73543feb4b35bba24c664ce0687fa41cf897a5d0378cfb236c29" gracePeriod=30
Sep 11 13:11:47.783613 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:47.783559    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:47.942668 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:47.942632    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746d6a16-775e-43ba-b4d7-42de068b6dbe-metrics-certs\") pod \"746d6a16-775e-43ba-b4d7-42de068b6dbe\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") "
Sep 11 13:11:47.942854 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:47.942700    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/746d6a16-775e-43ba-b4d7-42de068b6dbe-maas-api-tls\") pod \"746d6a16-775e-43ba-b4d7-42de068b6dbe\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") "
Sep 11 13:11:47.942854 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:47.942773    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnkjv\" (UniqueName: \"kubernetes.io/projected/746d6a16-775e-43ba-b4d7-42de068b6dbe-kube-api-access-xnkjv\") pod \"746d6a16-775e-43ba-b4d7-42de068b6dbe\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") "
Sep 11 13:11:47.942854 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:47.942832    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/746d6a16-775e-43ba-b4d7-42de068b6dbe-openshift-service-ca\") pod \"746d6a16-775e-43ba-b4d7-42de068b6dbe\" (UID: \"746d6a16-775e-43ba-b4d7-42de068b6dbe\") "
Sep 11 13:11:47.943230 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:47.943204    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/746d6a16-775e-43ba-b4d7-42de068b6dbe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "746d6a16-775e-43ba-b4d7-42de068b6dbe" (UID: "746d6a16-775e-43ba-b4d7-42de068b6dbe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:11:47.944853 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:47.944817    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/746d6a16-775e-43ba-b4d7-42de068b6dbe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "746d6a16-775e-43ba-b4d7-42de068b6dbe" (UID: "746d6a16-775e-43ba-b4d7-42de068b6dbe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:11:47.945275 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:47.945253    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/746d6a16-775e-43ba-b4d7-42de068b6dbe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "746d6a16-775e-43ba-b4d7-42de068b6dbe" (UID: "746d6a16-775e-43ba-b4d7-42de068b6dbe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:11:47.945362 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:47.945271    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/746d6a16-775e-43ba-b4d7-42de068b6dbe-kube-api-access-xnkjv" (OuterVolumeSpecName: "kube-api-access-xnkjv") pod "746d6a16-775e-43ba-b4d7-42de068b6dbe" (UID: "746d6a16-775e-43ba-b4d7-42de068b6dbe"). InnerVolumeSpecName "kube-api-access-xnkjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:11:48.044310 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:48.044237    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/746d6a16-775e-43ba-b4d7-42de068b6dbe-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:48.044310 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:48.044266    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnkjv\" (UniqueName: \"kubernetes.io/projected/746d6a16-775e-43ba-b4d7-42de068b6dbe-kube-api-access-xnkjv\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:48.044310 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:48.044275    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/746d6a16-775e-43ba-b4d7-42de068b6dbe-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:48.044310 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:48.044284    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/746d6a16-775e-43ba-b4d7-42de068b6dbe-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:11:48.467307 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:48.467224    2582 generic.go:356] "Generic (PLEG): container finished" podID="746d6a16-775e-43ba-b4d7-42de068b6dbe" containerID="481d20c916cb73543feb4b35bba24c664ce0687fa41cf897a5d0378cfb236c29" exitCode=0
Sep 11 13:11:48.467307 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:48.467271    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"
Sep 11 13:11:48.467307 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:48.467287    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b" event={"ID":"746d6a16-775e-43ba-b4d7-42de068b6dbe","Type":"ContainerDied","Data":"481d20c916cb73543feb4b35bba24c664ce0687fa41cf897a5d0378cfb236c29"}
Sep 11 13:11:48.467615 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:48.467328    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b" event={"ID":"746d6a16-775e-43ba-b4d7-42de068b6dbe","Type":"ContainerDied","Data":"1b3e82530e5c6de2b5b5a52067e77f497337f32880ad630ebe82b93c3ba3b55d"}
Sep 11 13:11:48.467615 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:48.467350    2582 scope.go:117] "RemoveContainer" containerID="481d20c916cb73543feb4b35bba24c664ce0687fa41cf897a5d0378cfb236c29"
Sep 11 13:11:48.478408 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:48.478386    2582 scope.go:117] "RemoveContainer" containerID="481d20c916cb73543feb4b35bba24c664ce0687fa41cf897a5d0378cfb236c29"
Sep 11 13:11:48.478730 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:11:48.478712    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"481d20c916cb73543feb4b35bba24c664ce0687fa41cf897a5d0378cfb236c29\": container with ID starting with 481d20c916cb73543feb4b35bba24c664ce0687fa41cf897a5d0378cfb236c29 not found: ID does not exist" containerID="481d20c916cb73543feb4b35bba24c664ce0687fa41cf897a5d0378cfb236c29"
Sep 11 13:11:48.478807 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:48.478738    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"481d20c916cb73543feb4b35bba24c664ce0687fa41cf897a5d0378cfb236c29"} err="failed to get container status \"481d20c916cb73543feb4b35bba24c664ce0687fa41cf897a5d0378cfb236c29\": rpc error: code = NotFound desc = could not find container \"481d20c916cb73543feb4b35bba24c664ce0687fa41cf897a5d0378cfb236c29\": container with ID starting with 481d20c916cb73543feb4b35bba24c664ce0687fa41cf897a5d0378cfb236c29 not found: ID does not exist"
Sep 11 13:11:48.509593 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:48.509563    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"]
Sep 11 13:11:48.521910 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:48.521884    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ecc2b9-6b8c7565cc-tft5b"]
Sep 11 13:11:50.477371 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:11:50.477335    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="746d6a16-775e-43ba-b4d7-42de068b6dbe" path="/var/lib/kubelet/pods/746d6a16-775e-43ba-b4d7-42de068b6dbe/volumes"
Sep 11 13:12:44.438598 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.438565    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"]
Sep 11 13:12:44.439103 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.438764    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b" podUID="0ce32a64-e063-46a0-a11a-f94314f18528" containerName="maas-api" containerID="cri-o://396c278ad7d48dd40a01335fc9e829117eddc951815def2541cbd553f9d11aa5" gracePeriod=30
Sep 11 13:12:44.713955 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.713931    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:12:44.773185 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.773154    2582 generic.go:356] "Generic (PLEG): container finished" podID="0ce32a64-e063-46a0-a11a-f94314f18528" containerID="396c278ad7d48dd40a01335fc9e829117eddc951815def2541cbd553f9d11aa5" exitCode=0
Sep 11 13:12:44.773325 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.773194    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"
Sep 11 13:12:44.773325 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.773238    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b" event={"ID":"0ce32a64-e063-46a0-a11a-f94314f18528","Type":"ContainerDied","Data":"396c278ad7d48dd40a01335fc9e829117eddc951815def2541cbd553f9d11aa5"}
Sep 11 13:12:44.773325 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.773289    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b" event={"ID":"0ce32a64-e063-46a0-a11a-f94314f18528","Type":"ContainerDied","Data":"4fa3cab930e585e14d21f981cb7bfe9b78a6635e90e5eca8cb46136022c388c9"}
Sep 11 13:12:44.773325 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.773317    2582 scope.go:117] "RemoveContainer" containerID="396c278ad7d48dd40a01335fc9e829117eddc951815def2541cbd553f9d11aa5"
Sep 11 13:12:44.781792 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.781738    2582 scope.go:117] "RemoveContainer" containerID="396c278ad7d48dd40a01335fc9e829117eddc951815def2541cbd553f9d11aa5"
Sep 11 13:12:44.782044 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:12:44.782023    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"396c278ad7d48dd40a01335fc9e829117eddc951815def2541cbd553f9d11aa5\": container with ID starting with 396c278ad7d48dd40a01335fc9e829117eddc951815def2541cbd553f9d11aa5 not found: ID does not exist" containerID="396c278ad7d48dd40a01335fc9e829117eddc951815def2541cbd553f9d11aa5"
Sep 11 13:12:44.782137 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.782052    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"396c278ad7d48dd40a01335fc9e829117eddc951815def2541cbd553f9d11aa5"} err="failed to get container status \"396c278ad7d48dd40a01335fc9e829117eddc951815def2541cbd553f9d11aa5\": rpc error: code = NotFound desc = could not find container \"396c278ad7d48dd40a01335fc9e829117eddc951815def2541cbd553f9d11aa5\": container with ID starting with 396c278ad7d48dd40a01335fc9e829117eddc951815def2541cbd553f9d11aa5 not found: ID does not exist"
Sep 11 13:12:44.845930 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.845904    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ce32a64-e063-46a0-a11a-f94314f18528-metrics-certs\") pod \"0ce32a64-e063-46a0-a11a-f94314f18528\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") "
Sep 11 13:12:44.846057 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.845951    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmrv7\" (UniqueName: \"kubernetes.io/projected/0ce32a64-e063-46a0-a11a-f94314f18528-kube-api-access-nmrv7\") pod \"0ce32a64-e063-46a0-a11a-f94314f18528\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") "
Sep 11 13:12:44.846123 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.846065    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ce32a64-e063-46a0-a11a-f94314f18528-maas-api-tls\") pod \"0ce32a64-e063-46a0-a11a-f94314f18528\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") "
Sep 11 13:12:44.846123 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.846101    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ce32a64-e063-46a0-a11a-f94314f18528-openshift-service-ca\") pod \"0ce32a64-e063-46a0-a11a-f94314f18528\" (UID: \"0ce32a64-e063-46a0-a11a-f94314f18528\") "
Sep 11 13:12:44.846559 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.846532    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ce32a64-e063-46a0-a11a-f94314f18528-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0ce32a64-e063-46a0-a11a-f94314f18528" (UID: "0ce32a64-e063-46a0-a11a-f94314f18528"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:12:44.848010 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.847971    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ce32a64-e063-46a0-a11a-f94314f18528-kube-api-access-nmrv7" (OuterVolumeSpecName: "kube-api-access-nmrv7") pod "0ce32a64-e063-46a0-a11a-f94314f18528" (UID: "0ce32a64-e063-46a0-a11a-f94314f18528"). InnerVolumeSpecName "kube-api-access-nmrv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:12:44.848130 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.848064    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ce32a64-e063-46a0-a11a-f94314f18528-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0ce32a64-e063-46a0-a11a-f94314f18528" (UID: "0ce32a64-e063-46a0-a11a-f94314f18528"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:12:44.848130 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.848110    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ce32a64-e063-46a0-a11a-f94314f18528-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0ce32a64-e063-46a0-a11a-f94314f18528" (UID: "0ce32a64-e063-46a0-a11a-f94314f18528"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:12:44.947399 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.947338    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ce32a64-e063-46a0-a11a-f94314f18528-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:12:44.947399 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.947364    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ce32a64-e063-46a0-a11a-f94314f18528-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:12:44.947399 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.947377    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ce32a64-e063-46a0-a11a-f94314f18528-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:12:44.947399 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:44.947389    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmrv7\" (UniqueName: \"kubernetes.io/projected/0ce32a64-e063-46a0-a11a-f94314f18528-kube-api-access-nmrv7\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:12:45.108202 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:45.108176    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"]
Sep 11 13:12:45.112614 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:45.112593    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-528ff6-7b77bbb57f-h2r2b"]
Sep 11 13:12:46.476800 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:12:46.476768    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ce32a64-e063-46a0-a11a-f94314f18528" path="/var/lib/kubelet/pods/0ce32a64-e063-46a0-a11a-f94314f18528/volumes"
Sep 11 13:13:06.872354 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872320    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"]
Sep 11 13:13:06.872817 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872754    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="746d6a16-775e-43ba-b4d7-42de068b6dbe" containerName="maas-api"
Sep 11 13:13:06.872817 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872766    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="746d6a16-775e-43ba-b4d7-42de068b6dbe" containerName="maas-api"
Sep 11 13:13:06.872817 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872778    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" containerName="istio-proxy"
Sep 11 13:13:06.872817 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872784    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" containerName="istio-proxy"
Sep 11 13:13:06.872817 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872795    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" containerName="istio-proxy"
Sep 11 13:13:06.872817 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872800    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" containerName="istio-proxy"
Sep 11 13:13:06.872817 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872818    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ce32a64-e063-46a0-a11a-f94314f18528" containerName="maas-api"
Sep 11 13:13:06.873035 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872823    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ce32a64-e063-46a0-a11a-f94314f18528" containerName="maas-api"
Sep 11 13:13:06.873035 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872829    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df191588-26f7-4edc-aeff-2e1bc15d9181" containerName="maas-api"
Sep 11 13:13:06.873035 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872834    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="df191588-26f7-4edc-aeff-2e1bc15d9181" containerName="maas-api"
Sep 11 13:13:06.873035 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872898    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ce32a64-e063-46a0-a11a-f94314f18528" containerName="maas-api"
Sep 11 13:13:06.873035 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872904    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" containerName="istio-proxy"
Sep 11 13:13:06.873035 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872910    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="df191588-26f7-4edc-aeff-2e1bc15d9181" containerName="maas-api"
Sep 11 13:13:06.873035 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872917    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="647ccdc0-d78b-4c51-9d2d-d17d981e6bad" containerName="istio-proxy"
Sep 11 13:13:06.873035 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.872922    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="746d6a16-775e-43ba-b4d7-42de068b6dbe" containerName="maas-api"
Sep 11 13:13:06.877500 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.877482    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:06.885027 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.885005    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1e58f711-serving-cert\""
Sep 11 13:13:06.893200 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.893175    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"]
Sep 11 13:13:06.932121 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.932099    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e14a081-c903-4bae-a8a6-59d014dd070c-openshift-service-ca\") pod \"maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:06.932245 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.932215    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e14a081-c903-4bae-a8a6-59d014dd070c-maas-api-tls\") pod \"maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:06.932307 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.932250    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e14a081-c903-4bae-a8a6-59d014dd070c-metrics-certs\") pod \"maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:06.932307 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:06.932299    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h46nj\" (UniqueName: \"kubernetes.io/projected/1e14a081-c903-4bae-a8a6-59d014dd070c-kube-api-access-h46nj\") pod \"maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:07.032877 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:07.032851    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e14a081-c903-4bae-a8a6-59d014dd070c-openshift-service-ca\") pod \"maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:07.032983 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:07.032908    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e14a081-c903-4bae-a8a6-59d014dd070c-maas-api-tls\") pod \"maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:07.032983 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:07.032927    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e14a081-c903-4bae-a8a6-59d014dd070c-metrics-certs\") pod \"maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:07.032983 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:07.032954    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h46nj\" (UniqueName: \"kubernetes.io/projected/1e14a081-c903-4bae-a8a6-59d014dd070c-kube-api-access-h46nj\") pod \"maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:07.033506 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:07.033481    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e14a081-c903-4bae-a8a6-59d014dd070c-openshift-service-ca\") pod \"maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:07.035404 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:07.035379    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e14a081-c903-4bae-a8a6-59d014dd070c-metrics-certs\") pod \"maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:07.035404 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:07.035398    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e14a081-c903-4bae-a8a6-59d014dd070c-maas-api-tls\") pod \"maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:07.050791 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:07.050771    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h46nj\" (UniqueName: \"kubernetes.io/projected/1e14a081-c903-4bae-a8a6-59d014dd070c-kube-api-access-h46nj\") pod \"maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:07.189574 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:07.189511    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:07.494338 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:07.494235    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"]
Sep 11 13:13:07.497233 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:13:07.497205    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e14a081_c903_4bae_a8a6_59d014dd070c.slice/crio-6de6b8b5c74c4cde19ee025e203b22a8079b2892cefd7153c1ef431aeff0d972 WatchSource:0}: Error finding container 6de6b8b5c74c4cde19ee025e203b22a8079b2892cefd7153c1ef431aeff0d972: Status 404 returned error can't find the container with id 6de6b8b5c74c4cde19ee025e203b22a8079b2892cefd7153c1ef431aeff0d972
Sep 11 13:13:07.498686 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:07.498656    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:13:07.895525 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:07.895451    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92" event={"ID":"1e14a081-c903-4bae-a8a6-59d014dd070c","Type":"ContainerStarted","Data":"6de6b8b5c74c4cde19ee025e203b22a8079b2892cefd7153c1ef431aeff0d972"}
Sep 11 13:13:08.378394 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.378360    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"]
Sep 11 13:13:08.378592 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.378569    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg" podUID="a391ffb7-78cd-44cb-b082-e20d1af2d410" containerName="maas-api" containerID="cri-o://692ab53576dcfb7ba7b5d1443b0fdbd07ed274b557e3b1ace2a91e69bd3c41c2" gracePeriod=30
Sep 11 13:13:08.411501 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.411469    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg" podUID="a391ffb7-78cd-44cb-b082-e20d1af2d410" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.49:8443/health\": dial tcp 10.134.0.49:8443: connect: connection refused"
Sep 11 13:13:08.643528 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.643466    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:13:08.747965 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.747927    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a391ffb7-78cd-44cb-b082-e20d1af2d410-maas-api-tls\") pod \"a391ffb7-78cd-44cb-b082-e20d1af2d410\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") "
Sep 11 13:13:08.748154 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.748011    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a391ffb7-78cd-44cb-b082-e20d1af2d410-metrics-certs\") pod \"a391ffb7-78cd-44cb-b082-e20d1af2d410\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") "
Sep 11 13:13:08.748154 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.748030    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a391ffb7-78cd-44cb-b082-e20d1af2d410-openshift-service-ca\") pod \"a391ffb7-78cd-44cb-b082-e20d1af2d410\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") "
Sep 11 13:13:08.748154 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.748095    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqdw6\" (UniqueName: \"kubernetes.io/projected/a391ffb7-78cd-44cb-b082-e20d1af2d410-kube-api-access-sqdw6\") pod \"a391ffb7-78cd-44cb-b082-e20d1af2d410\" (UID: \"a391ffb7-78cd-44cb-b082-e20d1af2d410\") "
Sep 11 13:13:08.748483 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.748443    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a391ffb7-78cd-44cb-b082-e20d1af2d410-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a391ffb7-78cd-44cb-b082-e20d1af2d410" (UID: "a391ffb7-78cd-44cb-b082-e20d1af2d410"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:13:08.750112 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.750091    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a391ffb7-78cd-44cb-b082-e20d1af2d410-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a391ffb7-78cd-44cb-b082-e20d1af2d410" (UID: "a391ffb7-78cd-44cb-b082-e20d1af2d410"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:13:08.750199 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.750120    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a391ffb7-78cd-44cb-b082-e20d1af2d410-kube-api-access-sqdw6" (OuterVolumeSpecName: "kube-api-access-sqdw6") pod "a391ffb7-78cd-44cb-b082-e20d1af2d410" (UID: "a391ffb7-78cd-44cb-b082-e20d1af2d410"). InnerVolumeSpecName "kube-api-access-sqdw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:13:08.750199 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.750166    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a391ffb7-78cd-44cb-b082-e20d1af2d410-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a391ffb7-78cd-44cb-b082-e20d1af2d410" (UID: "a391ffb7-78cd-44cb-b082-e20d1af2d410"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:13:08.849090 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.849056    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a391ffb7-78cd-44cb-b082-e20d1af2d410-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:13:08.849090 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.849084    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a391ffb7-78cd-44cb-b082-e20d1af2d410-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:13:08.849090 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.849094    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqdw6\" (UniqueName: \"kubernetes.io/projected/a391ffb7-78cd-44cb-b082-e20d1af2d410-kube-api-access-sqdw6\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:13:08.849370 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.849102    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a391ffb7-78cd-44cb-b082-e20d1af2d410-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:13:08.902614 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.902529    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92" event={"ID":"1e14a081-c903-4bae-a8a6-59d014dd070c","Type":"ContainerStarted","Data":"3a203be5325d10bee69dbf089896fe6aba5c4605d9b82b09b0943466b3a6ce89"}
Sep 11 13:13:08.903015 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.902641    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:08.904076 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.904049    2582 generic.go:356] "Generic (PLEG): container finished" podID="a391ffb7-78cd-44cb-b082-e20d1af2d410" containerID="692ab53576dcfb7ba7b5d1443b0fdbd07ed274b557e3b1ace2a91e69bd3c41c2" exitCode=0
Sep 11 13:13:08.904186 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.904089    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"
Sep 11 13:13:08.904186 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.904084    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg" event={"ID":"a391ffb7-78cd-44cb-b082-e20d1af2d410","Type":"ContainerDied","Data":"692ab53576dcfb7ba7b5d1443b0fdbd07ed274b557e3b1ace2a91e69bd3c41c2"}
Sep 11 13:13:08.904186 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.904176    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg" event={"ID":"a391ffb7-78cd-44cb-b082-e20d1af2d410","Type":"ContainerDied","Data":"0d8f63f2acb0366105ab1aa0786ef009a2a6c03b89ac5798c9bc80f27e60eca3"}
Sep 11 13:13:08.904297 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.904202    2582 scope.go:117] "RemoveContainer" containerID="692ab53576dcfb7ba7b5d1443b0fdbd07ed274b557e3b1ace2a91e69bd3c41c2"
Sep 11 13:13:08.913511 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.913494    2582 scope.go:117] "RemoveContainer" containerID="692ab53576dcfb7ba7b5d1443b0fdbd07ed274b557e3b1ace2a91e69bd3c41c2"
Sep 11 13:13:08.913811 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:13:08.913791    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"692ab53576dcfb7ba7b5d1443b0fdbd07ed274b557e3b1ace2a91e69bd3c41c2\": container with ID starting with 692ab53576dcfb7ba7b5d1443b0fdbd07ed274b557e3b1ace2a91e69bd3c41c2 not found: ID does not exist" containerID="692ab53576dcfb7ba7b5d1443b0fdbd07ed274b557e3b1ace2a91e69bd3c41c2"
Sep 11 13:13:08.913873 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.913819    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"692ab53576dcfb7ba7b5d1443b0fdbd07ed274b557e3b1ace2a91e69bd3c41c2"} err="failed to get container status \"692ab53576dcfb7ba7b5d1443b0fdbd07ed274b557e3b1ace2a91e69bd3c41c2\": rpc error: code = NotFound desc = could not find container \"692ab53576dcfb7ba7b5d1443b0fdbd07ed274b557e3b1ace2a91e69bd3c41c2\": container with ID starting with 692ab53576dcfb7ba7b5d1443b0fdbd07ed274b557e3b1ace2a91e69bd3c41c2 not found: ID does not exist"
Sep 11 13:13:08.938044 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.938005    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92" podStartSLOduration=2.53926899 podStartE2EDuration="2.93799286s" totalImagesPullingTime="398.72387ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:13:08.936038989 +0000 UTC m=+1042.875740343" watchObservedRunningTime="2026-09-11 13:13:08.93799286 +0000 UTC m=+1042.877694193"
Sep 11 13:13:08.976624 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.976600    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"]
Sep 11 13:13:08.993490 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:08.993465    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6ba91-67b76f76f7-98bpg"]
Sep 11 13:13:10.476643 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:10.476613    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a391ffb7-78cd-44cb-b082-e20d1af2d410" path="/var/lib/kubelet/pods/a391ffb7-78cd-44cb-b082-e20d1af2d410/volumes"
Sep 11 13:13:14.915650 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:14.915616    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:27.526972 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.526924    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"]
Sep 11 13:13:27.527459 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.527442    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a391ffb7-78cd-44cb-b082-e20d1af2d410" containerName="maas-api"
Sep 11 13:13:27.527518 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.527461    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a391ffb7-78cd-44cb-b082-e20d1af2d410" containerName="maas-api"
Sep 11 13:13:27.527572 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.527537    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a391ffb7-78cd-44cb-b082-e20d1af2d410" containerName="maas-api"
Sep 11 13:13:27.530704 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.530689    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:27.534389 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.534367    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-1841da-serving-cert\""
Sep 11 13:13:27.558540 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.558514    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"]
Sep 11 13:13:27.625971 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.625936    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:27.626141 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.625997    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:27.626141 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.626067    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kzx8\" (UniqueName: \"kubernetes.io/projected/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-kube-api-access-6kzx8\") pod \"maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:27.626141 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.626135    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:27.727591 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.727549    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:27.727735 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.727608    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kzx8\" (UniqueName: \"kubernetes.io/projected/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-kube-api-access-6kzx8\") pod \"maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:27.727735 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.727656    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:27.727735 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.727722    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:27.728410 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.728386    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:27.730206 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.730180    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:27.730590 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.730568    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:27.749045 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.749018    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kzx8\" (UniqueName: \"kubernetes.io/projected/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-kube-api-access-6kzx8\") pod \"maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:27.841170 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:27.841089    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:28.134974 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:28.134949    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"]
Sep 11 13:13:28.137057 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:13:28.137031    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29e1a8f2_e0b4_4679_acf4_1dadecfd0133.slice/crio-2e9a441f5d2cdc20d4b151501b1cc666072c1870a9dd5d9e59a385347992eded WatchSource:0}: Error finding container 2e9a441f5d2cdc20d4b151501b1cc666072c1870a9dd5d9e59a385347992eded: Status 404 returned error can't find the container with id 2e9a441f5d2cdc20d4b151501b1cc666072c1870a9dd5d9e59a385347992eded
Sep 11 13:13:29.010181 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:29.010144    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j" event={"ID":"29e1a8f2-e0b4-4679-acf4-1dadecfd0133","Type":"ContainerStarted","Data":"f3be677a00e3d05dfb1255682ce9ed992ac9170161839fa56783f217ae78cc3b"}
Sep 11 13:13:29.010619 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:29.010206    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j" event={"ID":"29e1a8f2-e0b4-4679-acf4-1dadecfd0133","Type":"ContainerStarted","Data":"2e9a441f5d2cdc20d4b151501b1cc666072c1870a9dd5d9e59a385347992eded"}
Sep 11 13:13:29.010619 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:29.010333    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:29.091787 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:29.091724    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j" podStartSLOduration=1.6411953750000001 podStartE2EDuration="2.091708542s" totalImagesPullingTime="450.513167ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:13:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:13:29.090406739 +0000 UTC m=+1063.030108072" watchObservedRunningTime="2026-09-11 13:13:29.091708542 +0000 UTC m=+1063.031409873"
Sep 11 13:13:35.021346 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:35.021315    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:13:51.609636 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.609605    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"]
Sep 11 13:13:51.610201 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.609891    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92" podUID="1e14a081-c903-4bae-a8a6-59d014dd070c" containerName="maas-api" containerID="cri-o://3a203be5325d10bee69dbf089896fe6aba5c4605d9b82b09b0943466b3a6ce89" gracePeriod=30
Sep 11 13:13:51.867960 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.867890    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:51.949842 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.949808    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h46nj\" (UniqueName: \"kubernetes.io/projected/1e14a081-c903-4bae-a8a6-59d014dd070c-kube-api-access-h46nj\") pod \"1e14a081-c903-4bae-a8a6-59d014dd070c\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") "
Sep 11 13:13:51.949842 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.949847    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e14a081-c903-4bae-a8a6-59d014dd070c-metrics-certs\") pod \"1e14a081-c903-4bae-a8a6-59d014dd070c\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") "
Sep 11 13:13:51.950038 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.949898    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e14a081-c903-4bae-a8a6-59d014dd070c-maas-api-tls\") pod \"1e14a081-c903-4bae-a8a6-59d014dd070c\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") "
Sep 11 13:13:51.950038 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.949922    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e14a081-c903-4bae-a8a6-59d014dd070c-openshift-service-ca\") pod \"1e14a081-c903-4bae-a8a6-59d014dd070c\" (UID: \"1e14a081-c903-4bae-a8a6-59d014dd070c\") "
Sep 11 13:13:51.950357 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.950332    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e14a081-c903-4bae-a8a6-59d014dd070c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1e14a081-c903-4bae-a8a6-59d014dd070c" (UID: "1e14a081-c903-4bae-a8a6-59d014dd070c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:13:51.958909 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.958869    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e14a081-c903-4bae-a8a6-59d014dd070c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1e14a081-c903-4bae-a8a6-59d014dd070c" (UID: "1e14a081-c903-4bae-a8a6-59d014dd070c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:13:51.959027 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.958904    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e14a081-c903-4bae-a8a6-59d014dd070c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1e14a081-c903-4bae-a8a6-59d014dd070c" (UID: "1e14a081-c903-4bae-a8a6-59d014dd070c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:13:51.959027 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.958940    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e14a081-c903-4bae-a8a6-59d014dd070c-kube-api-access-h46nj" (OuterVolumeSpecName: "kube-api-access-h46nj") pod "1e14a081-c903-4bae-a8a6-59d014dd070c" (UID: "1e14a081-c903-4bae-a8a6-59d014dd070c"). InnerVolumeSpecName "kube-api-access-h46nj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:13:51.970280 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.970256    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"]
Sep 11 13:13:51.970793 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.970776    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e14a081-c903-4bae-a8a6-59d014dd070c" containerName="maas-api"
Sep 11 13:13:51.970882 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.970795    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e14a081-c903-4bae-a8a6-59d014dd070c" containerName="maas-api"
Sep 11 13:13:51.970970 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.970957    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e14a081-c903-4bae-a8a6-59d014dd070c" containerName="maas-api"
Sep 11 13:13:51.974645 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.974628    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:51.977672 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.977651    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-82825f-serving-cert\""
Sep 11 13:13:51.997447 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:51.997409    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"]
Sep 11 13:13:52.050727 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.050702    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afb691b2-b7ec-4a41-89b4-850561e49836-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:52.050853 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.050737    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afb691b2-b7ec-4a41-89b4-850561e49836-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:52.050853 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.050818    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9ndf\" (UniqueName: \"kubernetes.io/projected/afb691b2-b7ec-4a41-89b4-850561e49836-kube-api-access-q9ndf\") pod \"maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:52.051037 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.050949    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afb691b2-b7ec-4a41-89b4-850561e49836-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:52.051094 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.051048    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e14a081-c903-4bae-a8a6-59d014dd070c-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:13:52.051094 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.051063    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e14a081-c903-4bae-a8a6-59d014dd070c-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:13:52.051094 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.051072    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e14a081-c903-4bae-a8a6-59d014dd070c-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:13:52.051094 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.051082    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h46nj\" (UniqueName: \"kubernetes.io/projected/1e14a081-c903-4bae-a8a6-59d014dd070c-kube-api-access-h46nj\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:13:52.139291 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.139211    2582 generic.go:356] "Generic (PLEG): container finished" podID="1e14a081-c903-4bae-a8a6-59d014dd070c" containerID="3a203be5325d10bee69dbf089896fe6aba5c4605d9b82b09b0943466b3a6ce89" exitCode=0
Sep 11 13:13:52.139291 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.139264    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"
Sep 11 13:13:52.139510 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.139295    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92" event={"ID":"1e14a081-c903-4bae-a8a6-59d014dd070c","Type":"ContainerDied","Data":"3a203be5325d10bee69dbf089896fe6aba5c4605d9b82b09b0943466b3a6ce89"}
Sep 11 13:13:52.139510 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.139347    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92" event={"ID":"1e14a081-c903-4bae-a8a6-59d014dd070c","Type":"ContainerDied","Data":"6de6b8b5c74c4cde19ee025e203b22a8079b2892cefd7153c1ef431aeff0d972"}
Sep 11 13:13:52.139510 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.139370    2582 scope.go:117] "RemoveContainer" containerID="3a203be5325d10bee69dbf089896fe6aba5c4605d9b82b09b0943466b3a6ce89"
Sep 11 13:13:52.148169 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.148155    2582 scope.go:117] "RemoveContainer" containerID="3a203be5325d10bee69dbf089896fe6aba5c4605d9b82b09b0943466b3a6ce89"
Sep 11 13:13:52.148457 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:13:52.148409    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a203be5325d10bee69dbf089896fe6aba5c4605d9b82b09b0943466b3a6ce89\": container with ID starting with 3a203be5325d10bee69dbf089896fe6aba5c4605d9b82b09b0943466b3a6ce89 not found: ID does not exist" containerID="3a203be5325d10bee69dbf089896fe6aba5c4605d9b82b09b0943466b3a6ce89"
Sep 11 13:13:52.148532 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.148459    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a203be5325d10bee69dbf089896fe6aba5c4605d9b82b09b0943466b3a6ce89"} err="failed to get container status \"3a203be5325d10bee69dbf089896fe6aba5c4605d9b82b09b0943466b3a6ce89\": rpc error: code = NotFound desc = could not find container \"3a203be5325d10bee69dbf089896fe6aba5c4605d9b82b09b0943466b3a6ce89\": container with ID starting with 3a203be5325d10bee69dbf089896fe6aba5c4605d9b82b09b0943466b3a6ce89 not found: ID does not exist"
Sep 11 13:13:52.152337 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.151666    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afb691b2-b7ec-4a41-89b4-850561e49836-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:52.152337 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.151722    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afb691b2-b7ec-4a41-89b4-850561e49836-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:52.152337 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.151750    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afb691b2-b7ec-4a41-89b4-850561e49836-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:52.152337 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.151806    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9ndf\" (UniqueName: \"kubernetes.io/projected/afb691b2-b7ec-4a41-89b4-850561e49836-kube-api-access-q9ndf\") pod \"maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:52.152337 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:13:52.152096    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-serving-cert: secret "maas-api-e2e-shared-b-w3-82825f-serving-cert" not found
Sep 11 13:13:52.152337 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:13:52.152159    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb691b2-b7ec-4a41-89b4-850561e49836-maas-api-tls podName:afb691b2-b7ec-4a41-89b4-850561e49836 nodeName:}" failed. No retries permitted until 2026-09-11 13:13:52.652139625 +0000 UTC m=+1086.591840942 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/afb691b2-b7ec-4a41-89b4-850561e49836-maas-api-tls") pod "maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk" (UID: "afb691b2-b7ec-4a41-89b4-850561e49836") : secret "maas-api-e2e-shared-b-w3-82825f-serving-cert" not found
Sep 11 13:13:52.152747 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.152678    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afb691b2-b7ec-4a41-89b4-850561e49836-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:52.155064 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.155043    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afb691b2-b7ec-4a41-89b4-850561e49836-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:52.168475 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.168454    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9ndf\" (UniqueName: \"kubernetes.io/projected/afb691b2-b7ec-4a41-89b4-850561e49836-kube-api-access-q9ndf\") pod \"maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:52.179287 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.179265    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"]
Sep 11 13:13:52.193468 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.193447    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1e58f711-54fcbb54f4-c2s92"]
Sep 11 13:13:52.476379 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.476350    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e14a081-c903-4bae-a8a6-59d014dd070c" path="/var/lib/kubelet/pods/1e14a081-c903-4bae-a8a6-59d014dd070c/volumes"
Sep 11 13:13:52.657374 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.657339    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afb691b2-b7ec-4a41-89b4-850561e49836-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:52.659621 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.659589    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afb691b2-b7ec-4a41-89b4-850561e49836-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:52.889879 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:52.889806    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:53.183776 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:53.183698    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"]
Sep 11 13:13:53.189756 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:13:53.189703    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafb691b2_b7ec_4a41_89b4_850561e49836.slice/crio-3050111286b1e08a2b92ef8201423ba592f8a43b3193d452711f32269198688d WatchSource:0}: Error finding container 3050111286b1e08a2b92ef8201423ba592f8a43b3193d452711f32269198688d: Status 404 returned error can't find the container with id 3050111286b1e08a2b92ef8201423ba592f8a43b3193d452711f32269198688d
Sep 11 13:13:54.153592 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:54.153547    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk" event={"ID":"afb691b2-b7ec-4a41-89b4-850561e49836","Type":"ContainerStarted","Data":"d99b0653a1f800f7fb469f8a0a2f8a6b3c0468db60e538744e13f4077b63b5e9"}
Sep 11 13:13:54.154086 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:54.153618    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk" event={"ID":"afb691b2-b7ec-4a41-89b4-850561e49836","Type":"ContainerStarted","Data":"3050111286b1e08a2b92ef8201423ba592f8a43b3193d452711f32269198688d"}
Sep 11 13:13:54.154086 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:54.153646    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:13:54.181671 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:13:54.181622    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk" podStartSLOduration=2.78607342 podStartE2EDuration="3.18160689s" totalImagesPullingTime="395.53347ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:13:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:13:54.181301681 +0000 UTC m=+1088.121003022" watchObservedRunningTime="2026-09-11 13:13:54.18160689 +0000 UTC m=+1088.121308221"
Sep 11 13:14:00.165339 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:00.165311    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:14:23.686055 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:23.686018    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"]
Sep 11 13:14:23.689732 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:23.689709    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:23.697179 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:23.697158    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-4d7bb0-serving-cert\""
Sep 11 13:14:23.715617 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:23.715591    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"]
Sep 11 13:14:23.741259 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:23.741229    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:23.741374 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:23.741303    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:23.741374 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:23.741357    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:23.741483 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:23.741390    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6vzr\" (UniqueName: \"kubernetes.io/projected/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-kube-api-access-x6vzr\") pod \"maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:23.842778 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:23.842714    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:23.842778 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:23.842754    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:23.842915 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:23.842848    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6vzr\" (UniqueName: \"kubernetes.io/projected/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-kube-api-access-x6vzr\") pod \"maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:23.842915 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:14:23.842867    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-serving-cert: secret "maas-api-e2e-shared-a-w2-4d7bb0-serving-cert" not found
Sep 11 13:14:23.842915 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:23.842891    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:23.843017 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:14:23.842924    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-maas-api-tls podName:4dfc3b6e-172d-4d4e-82b1-1afbf65f3596 nodeName:}" failed. No retries permitted until 2026-09-11 13:14:24.342907779 +0000 UTC m=+1118.282609095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-maas-api-tls") pod "maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz" (UID: "4dfc3b6e-172d-4d4e-82b1-1afbf65f3596") : secret "maas-api-e2e-shared-a-w2-4d7bb0-serving-cert" not found
Sep 11 13:14:23.843552 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:23.843534    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:23.845300 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:23.845276    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:23.865746 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:23.865719    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6vzr\" (UniqueName: \"kubernetes.io/projected/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-kube-api-access-x6vzr\") pod \"maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:24.348239 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:24.348204    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:24.350563 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:24.350539    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:24.600008 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:24.599933    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:24.922308 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:24.922234    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"]
Sep 11 13:14:24.926558 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:14:24.926530    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dfc3b6e_172d_4d4e_82b1_1afbf65f3596.slice/crio-a69bee986eda4b8f6ed27b7d4bd410bb49ba56c0c7361f822e47c68a05e5003f WatchSource:0}: Error finding container a69bee986eda4b8f6ed27b7d4bd410bb49ba56c0c7361f822e47c68a05e5003f: Status 404 returned error can't find the container with id a69bee986eda4b8f6ed27b7d4bd410bb49ba56c0c7361f822e47c68a05e5003f
Sep 11 13:14:25.326521 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:25.326486    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz" event={"ID":"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596","Type":"ContainerStarted","Data":"a69bee986eda4b8f6ed27b7d4bd410bb49ba56c0c7361f822e47c68a05e5003f"}
Sep 11 13:14:26.332896 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:26.332854    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz" event={"ID":"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596","Type":"ContainerStarted","Data":"8035306a8128067cd88e25c3959e55b7b691fd28e7cbeac8e93a73d2ce88ad5c"}
Sep 11 13:14:26.333292 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:26.332974    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:26.391756 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:26.391707    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz" podStartSLOduration=2.77920209 podStartE2EDuration="3.391691903s" totalImagesPullingTime="612.489813ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:14:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:14:26.389806318 +0000 UTC m=+1120.329507650" watchObservedRunningTime="2026-09-11 13:14:26.391691903 +0000 UTC m=+1120.331393236"
Sep 11 13:14:32.345031 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:32.345000    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:14:46.588663 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:46.588635    2582 scope.go:117] "RemoveContainer" containerID="b1f8f347ea3fd32d81dc122d643a8d4e11874b6dd4dcedf7b7202f2c68e7c285"
Sep 11 13:14:58.657323 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:58.657287    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"]
Sep 11 13:14:58.662656 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:58.662635    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:14:58.672950 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:58.672929    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-f705b6-serving-cert\""
Sep 11 13:14:58.674372 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:58.674330    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466bb846-b3d5-41a9-b52e-22466cb40b29-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:14:58.674481 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:58.674447    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/466bb846-b3d5-41a9-b52e-22466cb40b29-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:14:58.674601 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:58.674587    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/466bb846-b3d5-41a9-b52e-22466cb40b29-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:14:58.674638 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:58.674613    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ldtm\" (UniqueName: \"kubernetes.io/projected/466bb846-b3d5-41a9-b52e-22466cb40b29-kube-api-access-4ldtm\") pod \"maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:14:58.695161 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:58.695136    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"]
Sep 11 13:14:58.775657 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:58.775627    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/466bb846-b3d5-41a9-b52e-22466cb40b29-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:14:58.775657 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:58.775659    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ldtm\" (UniqueName: \"kubernetes.io/projected/466bb846-b3d5-41a9-b52e-22466cb40b29-kube-api-access-4ldtm\") pod \"maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:14:58.775851 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:58.775694    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466bb846-b3d5-41a9-b52e-22466cb40b29-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:14:58.775851 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:58.775722    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/466bb846-b3d5-41a9-b52e-22466cb40b29-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:14:58.775851 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:14:58.775817    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-serving-cert: secret "maas-api-e2e-shared-b-w2-f705b6-serving-cert" not found
Sep 11 13:14:58.775967 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:14:58.775868    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/466bb846-b3d5-41a9-b52e-22466cb40b29-maas-api-tls podName:466bb846-b3d5-41a9-b52e-22466cb40b29 nodeName:}" failed. No retries permitted until 2026-09-11 13:14:59.275853594 +0000 UTC m=+1153.215554908 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/466bb846-b3d5-41a9-b52e-22466cb40b29-maas-api-tls") pod "maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5" (UID: "466bb846-b3d5-41a9-b52e-22466cb40b29") : secret "maas-api-e2e-shared-b-w2-f705b6-serving-cert" not found
Sep 11 13:14:58.776413 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:58.776393    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/466bb846-b3d5-41a9-b52e-22466cb40b29-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:14:58.778119 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:58.778098    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/466bb846-b3d5-41a9-b52e-22466cb40b29-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:14:58.802350 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:58.802320    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ldtm\" (UniqueName: \"kubernetes.io/projected/466bb846-b3d5-41a9-b52e-22466cb40b29-kube-api-access-4ldtm\") pod \"maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:14:59.279664 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:59.279630    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466bb846-b3d5-41a9-b52e-22466cb40b29-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:14:59.281945 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:59.281924    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466bb846-b3d5-41a9-b52e-22466cb40b29-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:14:59.572743 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:59.572659    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:14:59.882900 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:14:59.882875    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"]
Sep 11 13:14:59.885006 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:14:59.884982    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod466bb846_b3d5_41a9_b52e_22466cb40b29.slice/crio-5b9a0f17adbd006fd8a7d5c91e5278dc4b943dd2c84263a4c8481f52ce493a27 WatchSource:0}: Error finding container 5b9a0f17adbd006fd8a7d5c91e5278dc4b943dd2c84263a4c8481f52ce493a27: Status 404 returned error can't find the container with id 5b9a0f17adbd006fd8a7d5c91e5278dc4b943dd2c84263a4c8481f52ce493a27
Sep 11 13:15:00.525406 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:00.525368    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5" event={"ID":"466bb846-b3d5-41a9-b52e-22466cb40b29","Type":"ContainerStarted","Data":"c77aaac89e4d1650a9fefbb2c8fc5ce98755069f378fceb3959e80c5cc53f8fe"}
Sep 11 13:15:00.525600 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:00.525450    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5" event={"ID":"466bb846-b3d5-41a9-b52e-22466cb40b29","Type":"ContainerStarted","Data":"5b9a0f17adbd006fd8a7d5c91e5278dc4b943dd2c84263a4c8481f52ce493a27"}
Sep 11 13:15:00.525600 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:00.525479    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:15:00.581826 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:00.581729    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5" podStartSLOduration=2.14695607 podStartE2EDuration="2.581713369s" totalImagesPullingTime="434.757299ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:15:00.58153049 +0000 UTC m=+1154.521231822" watchObservedRunningTime="2026-09-11 13:15:00.581713369 +0000 UTC m=+1154.521414733"
Sep 11 13:15:06.535939 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:06.535908    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:15:40.445126 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:40.445093    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"]
Sep 11 13:15:40.457566 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:40.457537    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:40.462170 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:40.462141    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"]
Sep 11 13:15:40.468910 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:40.468713    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-13b2e851-serving-cert\""
Sep 11 13:15:40.571798 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:40.571771    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43241c70-5a79-4018-b92d-a760eb3c1124-maas-api-tls\") pod \"maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:40.571943 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:40.571860    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43241c70-5a79-4018-b92d-a760eb3c1124-openshift-service-ca\") pod \"maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:40.571943 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:40.571901    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmj5g\" (UniqueName: \"kubernetes.io/projected/43241c70-5a79-4018-b92d-a760eb3c1124-kube-api-access-fmj5g\") pod \"maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:40.572050 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:40.571976    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43241c70-5a79-4018-b92d-a760eb3c1124-metrics-certs\") pod \"maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:40.673358 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:40.673330    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43241c70-5a79-4018-b92d-a760eb3c1124-maas-api-tls\") pod \"maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:40.673558 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:40.673416    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43241c70-5a79-4018-b92d-a760eb3c1124-openshift-service-ca\") pod \"maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:40.673558 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:40.673485    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmj5g\" (UniqueName: \"kubernetes.io/projected/43241c70-5a79-4018-b92d-a760eb3c1124-kube-api-access-fmj5g\") pod \"maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:40.673558 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:15:40.673505    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-serving-cert: secret "maas-api-e2e-mt-13b2e851-serving-cert" not found
Sep 11 13:15:40.673558 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:40.673549    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43241c70-5a79-4018-b92d-a760eb3c1124-metrics-certs\") pod \"maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:40.673795 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:15:40.673592    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43241c70-5a79-4018-b92d-a760eb3c1124-maas-api-tls podName:43241c70-5a79-4018-b92d-a760eb3c1124 nodeName:}" failed. No retries permitted until 2026-09-11 13:15:41.173572993 +0000 UTC m=+1195.113274314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/43241c70-5a79-4018-b92d-a760eb3c1124-maas-api-tls") pod "maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s" (UID: "43241c70-5a79-4018-b92d-a760eb3c1124") : secret "maas-api-e2e-mt-13b2e851-serving-cert" not found
Sep 11 13:15:40.674130 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:40.674106    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43241c70-5a79-4018-b92d-a760eb3c1124-openshift-service-ca\") pod \"maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:40.675880 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:40.675846    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43241c70-5a79-4018-b92d-a760eb3c1124-metrics-certs\") pod \"maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:40.685015 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:40.684986    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmj5g\" (UniqueName: \"kubernetes.io/projected/43241c70-5a79-4018-b92d-a760eb3c1124-kube-api-access-fmj5g\") pod \"maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:41.178821 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:41.178785    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43241c70-5a79-4018-b92d-a760eb3c1124-maas-api-tls\") pod \"maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:41.181280 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:41.181253    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43241c70-5a79-4018-b92d-a760eb3c1124-maas-api-tls\") pod \"maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:41.378849 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:41.378819    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:41.666932 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:41.666902    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"]
Sep 11 13:15:41.667297 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:15:41.667201    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43241c70_5a79_4018_b92d_a760eb3c1124.slice/crio-8fb37baf8d9a643149028ed0bb81c024e8b166cc7686f7e312595d5f9ba6dca2 WatchSource:0}: Error finding container 8fb37baf8d9a643149028ed0bb81c024e8b166cc7686f7e312595d5f9ba6dca2: Status 404 returned error can't find the container with id 8fb37baf8d9a643149028ed0bb81c024e8b166cc7686f7e312595d5f9ba6dca2
Sep 11 13:15:41.748792 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:41.748763    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s" event={"ID":"43241c70-5a79-4018-b92d-a760eb3c1124","Type":"ContainerStarted","Data":"8fb37baf8d9a643149028ed0bb81c024e8b166cc7686f7e312595d5f9ba6dca2"}
Sep 11 13:15:42.756595 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:42.756549    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s" event={"ID":"43241c70-5a79-4018-b92d-a760eb3c1124","Type":"ContainerStarted","Data":"b98a51d073809065d190e348a296344611dd30d1e6786e7ee279a17a5a385994"}
Sep 11 13:15:42.757052 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:42.756610    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:15:42.783800 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:42.783740    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s" podStartSLOduration=2.332839931 podStartE2EDuration="2.78372122s" totalImagesPullingTime="450.881289ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:15:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:15:42.781698886 +0000 UTC m=+1196.721400212" watchObservedRunningTime="2026-09-11 13:15:42.78372122 +0000 UTC m=+1196.723422585"
Sep 11 13:15:48.767564 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:15:48.767531    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:17:02.218405 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:02.218333    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"]
Sep 11 13:17:02.218927 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:02.218570    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s" podUID="43241c70-5a79-4018-b92d-a760eb3c1124" containerName="maas-api" containerID="cri-o://b98a51d073809065d190e348a296344611dd30d1e6786e7ee279a17a5a385994" gracePeriod=30
Sep 11 13:17:02.484510 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:02.484489    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:17:02.581084 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:02.581060    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43241c70-5a79-4018-b92d-a760eb3c1124-maas-api-tls\") pod \"43241c70-5a79-4018-b92d-a760eb3c1124\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") "
Sep 11 13:17:02.581227 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:02.581125    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43241c70-5a79-4018-b92d-a760eb3c1124-openshift-service-ca\") pod \"43241c70-5a79-4018-b92d-a760eb3c1124\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") "
Sep 11 13:17:02.581227 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:02.581201    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43241c70-5a79-4018-b92d-a760eb3c1124-metrics-certs\") pod \"43241c70-5a79-4018-b92d-a760eb3c1124\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") "
Sep 11 13:17:02.581350 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:02.581296    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmj5g\" (UniqueName: \"kubernetes.io/projected/43241c70-5a79-4018-b92d-a760eb3c1124-kube-api-access-fmj5g\") pod \"43241c70-5a79-4018-b92d-a760eb3c1124\" (UID: \"43241c70-5a79-4018-b92d-a760eb3c1124\") "
Sep 11 13:17:02.581451 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:02.581402    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43241c70-5a79-4018-b92d-a760eb3c1124-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "43241c70-5a79-4018-b92d-a760eb3c1124" (UID: "43241c70-5a79-4018-b92d-a760eb3c1124"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:17:02.581634 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:02.581616    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43241c70-5a79-4018-b92d-a760eb3c1124-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:17:02.583389 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:02.583352    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43241c70-5a79-4018-b92d-a760eb3c1124-kube-api-access-fmj5g" (OuterVolumeSpecName: "kube-api-access-fmj5g") pod "43241c70-5a79-4018-b92d-a760eb3c1124" (UID: "43241c70-5a79-4018-b92d-a760eb3c1124"). InnerVolumeSpecName "kube-api-access-fmj5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:17:02.583681 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:02.583658    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43241c70-5a79-4018-b92d-a760eb3c1124-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "43241c70-5a79-4018-b92d-a760eb3c1124" (UID: "43241c70-5a79-4018-b92d-a760eb3c1124"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:17:02.583763 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:02.583679    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43241c70-5a79-4018-b92d-a760eb3c1124-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "43241c70-5a79-4018-b92d-a760eb3c1124" (UID: "43241c70-5a79-4018-b92d-a760eb3c1124"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:17:02.682884 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:02.682853    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43241c70-5a79-4018-b92d-a760eb3c1124-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:17:02.682884 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:02.682884    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmj5g\" (UniqueName: \"kubernetes.io/projected/43241c70-5a79-4018-b92d-a760eb3c1124-kube-api-access-fmj5g\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:17:02.683066 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:02.682898    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43241c70-5a79-4018-b92d-a760eb3c1124-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:17:03.199047 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:03.199012    2582 generic.go:356] "Generic (PLEG): container finished" podID="43241c70-5a79-4018-b92d-a760eb3c1124" containerID="b98a51d073809065d190e348a296344611dd30d1e6786e7ee279a17a5a385994" exitCode=0
Sep 11 13:17:03.199230 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:03.199065    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"
Sep 11 13:17:03.199230 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:03.199100    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s" event={"ID":"43241c70-5a79-4018-b92d-a760eb3c1124","Type":"ContainerDied","Data":"b98a51d073809065d190e348a296344611dd30d1e6786e7ee279a17a5a385994"}
Sep 11 13:17:03.199230 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:03.199161    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s" event={"ID":"43241c70-5a79-4018-b92d-a760eb3c1124","Type":"ContainerDied","Data":"8fb37baf8d9a643149028ed0bb81c024e8b166cc7686f7e312595d5f9ba6dca2"}
Sep 11 13:17:03.199230 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:03.199186    2582 scope.go:117] "RemoveContainer" containerID="b98a51d073809065d190e348a296344611dd30d1e6786e7ee279a17a5a385994"
Sep 11 13:17:03.208860 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:03.208844    2582 scope.go:117] "RemoveContainer" containerID="b98a51d073809065d190e348a296344611dd30d1e6786e7ee279a17a5a385994"
Sep 11 13:17:03.209136 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:17:03.209117    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b98a51d073809065d190e348a296344611dd30d1e6786e7ee279a17a5a385994\": container with ID starting with b98a51d073809065d190e348a296344611dd30d1e6786e7ee279a17a5a385994 not found: ID does not exist" containerID="b98a51d073809065d190e348a296344611dd30d1e6786e7ee279a17a5a385994"
Sep 11 13:17:03.209187 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:03.209143    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b98a51d073809065d190e348a296344611dd30d1e6786e7ee279a17a5a385994"} err="failed to get container status \"b98a51d073809065d190e348a296344611dd30d1e6786e7ee279a17a5a385994\": rpc error: code = NotFound desc = could not find container \"b98a51d073809065d190e348a296344611dd30d1e6786e7ee279a17a5a385994\": container with ID starting with b98a51d073809065d190e348a296344611dd30d1e6786e7ee279a17a5a385994 not found: ID does not exist"
Sep 11 13:17:03.304648 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:03.304621    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"]
Sep 11 13:17:03.312927 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:03.312902    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-13b2e851-bb6d44777-dhm7s"]
Sep 11 13:17:04.476615 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:04.476575    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43241c70-5a79-4018-b92d-a760eb3c1124" path="/var/lib/kubelet/pods/43241c70-5a79-4018-b92d-a760eb3c1124/volumes"
Sep 11 13:17:18.734862 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.734826    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"]
Sep 11 13:17:18.735358 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.735342    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43241c70-5a79-4018-b92d-a760eb3c1124" containerName="maas-api"
Sep 11 13:17:18.735358 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.735359    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="43241c70-5a79-4018-b92d-a760eb3c1124" containerName="maas-api"
Sep 11 13:17:18.735497 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.735451    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="43241c70-5a79-4018-b92d-a760eb3c1124" containerName="maas-api"
Sep 11 13:17:18.738683 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.738662    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.743896 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.743878    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-1841da\"/\"kube-root-ca.crt\""
Sep 11 13:17:18.743896 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.743886    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-1841da\"/\"default-dockercfg-9nsw8\""
Sep 11 13:17:18.744062 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.743888    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-1841da\"/\"sub-test-model-1841da-kserve-self-signed-certs\""
Sep 11 13:17:18.745633 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.745610    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-1841da\"/\"openshift-service-ca.crt\""
Sep 11 13:17:18.773011 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.772988    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"]
Sep 11 13:17:18.823394 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.823370    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-model-cache\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.823536 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.823412    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-dshm\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.823609 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.823547    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28330fd5-a7eb-4efe-bcd1-86efac40adb9-tls-certs\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.823667 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.823608    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rg8v\" (UniqueName: \"kubernetes.io/projected/28330fd5-a7eb-4efe-bcd1-86efac40adb9-kube-api-access-8rg8v\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.823716 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.823676    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-tmp-dir\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.823767 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.823741    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-home\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.924790 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.924761    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28330fd5-a7eb-4efe-bcd1-86efac40adb9-tls-certs\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.924911 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.924804    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rg8v\" (UniqueName: \"kubernetes.io/projected/28330fd5-a7eb-4efe-bcd1-86efac40adb9-kube-api-access-8rg8v\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.925646 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.925268    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-tmp-dir\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.925646 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.925358    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-home\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.925646 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.925437    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-model-cache\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.925646 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.925493    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-dshm\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.925646 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.925527    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-tmp-dir\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.926021 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.925798    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-home\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.926021 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.925881    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-model-cache\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.928269 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.928243    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-dshm\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.935532 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.935506    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28330fd5-a7eb-4efe-bcd1-86efac40adb9-tls-certs\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:18.936151 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:18.936126    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rg8v\" (UniqueName: \"kubernetes.io/projected/28330fd5-a7eb-4efe-bcd1-86efac40adb9-kube-api-access-8rg8v\") pod \"sub-test-model-1841da-kserve-56488b896f-7w5cz\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") " pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:19.048794 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:19.048735    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:19.339624 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:19.339597    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"]
Sep 11 13:17:19.341921 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:17:19.341886    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28330fd5_a7eb_4efe_bcd1_86efac40adb9.slice/crio-4ab7e6f64545e1a302d3c7572c6ab4cb7948f8079423666fd680cb2ff4ba6f78 WatchSource:0}: Error finding container 4ab7e6f64545e1a302d3c7572c6ab4cb7948f8079423666fd680cb2ff4ba6f78: Status 404 returned error can't find the container with id 4ab7e6f64545e1a302d3c7572c6ab4cb7948f8079423666fd680cb2ff4ba6f78
Sep 11 13:17:20.300317 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:20.300282    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz" event={"ID":"28330fd5-a7eb-4efe-bcd1-86efac40adb9","Type":"ContainerStarted","Data":"1cb7e4e49c8be6f757720a38365f6c02edfdf1ebf9e939a583ef62b82ce86512"}
Sep 11 13:17:20.300317 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:20.300322    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz" event={"ID":"28330fd5-a7eb-4efe-bcd1-86efac40adb9","Type":"ContainerStarted","Data":"4ab7e6f64545e1a302d3c7572c6ab4cb7948f8079423666fd680cb2ff4ba6f78"}
Sep 11 13:17:20.325920 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:20.325872    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz" podStartSLOduration=2.325856638 podStartE2EDuration="2.325856638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:17:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:17:20.323244972 +0000 UTC m=+1294.262946304" watchObservedRunningTime="2026-09-11 13:17:20.325856638 +0000 UTC m=+1294.265557993"
Sep 11 13:17:29.049887 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:29.049850    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:29.050294 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:29.049928    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:29.063212 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:29.063188    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:17:29.362296 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:17:29.362218    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:18:31.738757 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:31.738718    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"]
Sep 11 13:18:46.758668 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:46.758615    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz" podUID="28330fd5-a7eb-4efe-bcd1-86efac40adb9" containerName="main" containerID="cri-o://1cb7e4e49c8be6f757720a38365f6c02edfdf1ebf9e939a583ef62b82ce86512" gracePeriod=45
Sep 11 13:18:47.004491 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.004467    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:18:47.083484 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.083404    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-model-cache\") pod \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") "
Sep 11 13:18:47.083600 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.083511    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rg8v\" (UniqueName: \"kubernetes.io/projected/28330fd5-a7eb-4efe-bcd1-86efac40adb9-kube-api-access-8rg8v\") pod \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") "
Sep 11 13:18:47.083600 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.083536    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-tmp-dir\") pod \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") "
Sep 11 13:18:47.083600 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.083569    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-dshm\") pod \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") "
Sep 11 13:18:47.083763 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.083636    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28330fd5-a7eb-4efe-bcd1-86efac40adb9-tls-certs\") pod \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") "
Sep 11 13:18:47.083763 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.083676    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-home\") pod \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\" (UID: \"28330fd5-a7eb-4efe-bcd1-86efac40adb9\") "
Sep 11 13:18:47.083763 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.083632    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-model-cache" (OuterVolumeSpecName: "model-cache") pod "28330fd5-a7eb-4efe-bcd1-86efac40adb9" (UID: "28330fd5-a7eb-4efe-bcd1-86efac40adb9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:18:47.083916 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.083852    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "28330fd5-a7eb-4efe-bcd1-86efac40adb9" (UID: "28330fd5-a7eb-4efe-bcd1-86efac40adb9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:18:47.083998 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.083972    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-home" (OuterVolumeSpecName: "home") pod "28330fd5-a7eb-4efe-bcd1-86efac40adb9" (UID: "28330fd5-a7eb-4efe-bcd1-86efac40adb9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:18:47.084053 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.084036    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-model-cache\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:18:47.084096 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.084062    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-tmp-dir\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:18:47.085759 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.085732    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28330fd5-a7eb-4efe-bcd1-86efac40adb9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "28330fd5-a7eb-4efe-bcd1-86efac40adb9" (UID: "28330fd5-a7eb-4efe-bcd1-86efac40adb9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:18:47.085759 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.085748    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28330fd5-a7eb-4efe-bcd1-86efac40adb9-kube-api-access-8rg8v" (OuterVolumeSpecName: "kube-api-access-8rg8v") pod "28330fd5-a7eb-4efe-bcd1-86efac40adb9" (UID: "28330fd5-a7eb-4efe-bcd1-86efac40adb9"). InnerVolumeSpecName "kube-api-access-8rg8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:18:47.085938 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.085923    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-dshm" (OuterVolumeSpecName: "dshm") pod "28330fd5-a7eb-4efe-bcd1-86efac40adb9" (UID: "28330fd5-a7eb-4efe-bcd1-86efac40adb9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:18:47.185126 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.185101    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28330fd5-a7eb-4efe-bcd1-86efac40adb9-tls-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:18:47.185126 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.185124    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-home\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:18:47.185248 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.185134    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rg8v\" (UniqueName: \"kubernetes.io/projected/28330fd5-a7eb-4efe-bcd1-86efac40adb9-kube-api-access-8rg8v\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:18:47.185248 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.185147    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28330fd5-a7eb-4efe-bcd1-86efac40adb9-dshm\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:18:47.771573 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.771540    2582 generic.go:356] "Generic (PLEG): container finished" podID="28330fd5-a7eb-4efe-bcd1-86efac40adb9" containerID="1cb7e4e49c8be6f757720a38365f6c02edfdf1ebf9e939a583ef62b82ce86512" exitCode=0
Sep 11 13:18:47.772067 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.771588    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"
Sep 11 13:18:47.772067 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.771620    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz" event={"ID":"28330fd5-a7eb-4efe-bcd1-86efac40adb9","Type":"ContainerDied","Data":"1cb7e4e49c8be6f757720a38365f6c02edfdf1ebf9e939a583ef62b82ce86512"}
Sep 11 13:18:47.772067 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.771662    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz" event={"ID":"28330fd5-a7eb-4efe-bcd1-86efac40adb9","Type":"ContainerDied","Data":"4ab7e6f64545e1a302d3c7572c6ab4cb7948f8079423666fd680cb2ff4ba6f78"}
Sep 11 13:18:47.772067 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.771680    2582 scope.go:117] "RemoveContainer" containerID="1cb7e4e49c8be6f757720a38365f6c02edfdf1ebf9e939a583ef62b82ce86512"
Sep 11 13:18:47.782302 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.782283    2582 scope.go:117] "RemoveContainer" containerID="1cb7e4e49c8be6f757720a38365f6c02edfdf1ebf9e939a583ef62b82ce86512"
Sep 11 13:18:47.782610 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:18:47.782592    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cb7e4e49c8be6f757720a38365f6c02edfdf1ebf9e939a583ef62b82ce86512\": container with ID starting with 1cb7e4e49c8be6f757720a38365f6c02edfdf1ebf9e939a583ef62b82ce86512 not found: ID does not exist" containerID="1cb7e4e49c8be6f757720a38365f6c02edfdf1ebf9e939a583ef62b82ce86512"
Sep 11 13:18:47.782677 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.782617    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb7e4e49c8be6f757720a38365f6c02edfdf1ebf9e939a583ef62b82ce86512"} err="failed to get container status \"1cb7e4e49c8be6f757720a38365f6c02edfdf1ebf9e939a583ef62b82ce86512\": rpc error: code = NotFound desc = could not find container \"1cb7e4e49c8be6f757720a38365f6c02edfdf1ebf9e939a583ef62b82ce86512\": container with ID starting with 1cb7e4e49c8be6f757720a38365f6c02edfdf1ebf9e939a583ef62b82ce86512 not found: ID does not exist"
Sep 11 13:18:47.812400 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.812374    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"]
Sep 11 13:18:47.825246 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:47.825216    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1841da/sub-test-model-1841da-kserve-56488b896f-7w5cz"]
Sep 11 13:18:48.476801 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:18:48.476763    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28330fd5-a7eb-4efe-bcd1-86efac40adb9" path="/var/lib/kubelet/pods/28330fd5-a7eb-4efe-bcd1-86efac40adb9/volumes"
Sep 11 13:19:06.836681 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:06.836647    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"]
Sep 11 13:19:06.837332 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:06.837317    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28330fd5-a7eb-4efe-bcd1-86efac40adb9" containerName="main"
Sep 11 13:19:06.837385 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:06.837334    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="28330fd5-a7eb-4efe-bcd1-86efac40adb9" containerName="main"
Sep 11 13:19:06.837464 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:06.837452    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="28330fd5-a7eb-4efe-bcd1-86efac40adb9" containerName="main"
Sep 11 13:19:06.842272 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:06.842247    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:06.845479 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:06.845456    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-cd3c29-serving-cert\""
Sep 11 13:19:06.862696 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:06.862668    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"]
Sep 11 13:19:06.971191 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:06.971165    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf0b558b-4f01-49d5-97fc-419e9218340b-metrics-certs\") pod \"maas-api-e2e-api-a-cd3c29-5d96779975-55mdg\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:06.971345 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:06.971198    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxftq\" (UniqueName: \"kubernetes.io/projected/cf0b558b-4f01-49d5-97fc-419e9218340b-kube-api-access-bxftq\") pod \"maas-api-e2e-api-a-cd3c29-5d96779975-55mdg\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:06.971345 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:06.971247    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf0b558b-4f01-49d5-97fc-419e9218340b-openshift-service-ca\") pod \"maas-api-e2e-api-a-cd3c29-5d96779975-55mdg\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:06.971512 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:06.971368    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf0b558b-4f01-49d5-97fc-419e9218340b-maas-api-tls\") pod \"maas-api-e2e-api-a-cd3c29-5d96779975-55mdg\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:07.072248 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:07.072220    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf0b558b-4f01-49d5-97fc-419e9218340b-openshift-service-ca\") pod \"maas-api-e2e-api-a-cd3c29-5d96779975-55mdg\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:07.072358 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:07.072282    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf0b558b-4f01-49d5-97fc-419e9218340b-maas-api-tls\") pod \"maas-api-e2e-api-a-cd3c29-5d96779975-55mdg\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:07.072358 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:07.072335    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf0b558b-4f01-49d5-97fc-419e9218340b-metrics-certs\") pod \"maas-api-e2e-api-a-cd3c29-5d96779975-55mdg\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:07.072517 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:07.072363    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxftq\" (UniqueName: \"kubernetes.io/projected/cf0b558b-4f01-49d5-97fc-419e9218340b-kube-api-access-bxftq\") pod \"maas-api-e2e-api-a-cd3c29-5d96779975-55mdg\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:07.072517 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:19:07.072483    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-serving-cert: secret "maas-api-e2e-api-a-cd3c29-serving-cert" not found
Sep 11 13:19:07.072627 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:19:07.072556    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf0b558b-4f01-49d5-97fc-419e9218340b-maas-api-tls podName:cf0b558b-4f01-49d5-97fc-419e9218340b nodeName:}" failed. No retries permitted until 2026-09-11 13:19:07.572534896 +0000 UTC m=+1401.512236207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cf0b558b-4f01-49d5-97fc-419e9218340b-maas-api-tls") pod "maas-api-e2e-api-a-cd3c29-5d96779975-55mdg" (UID: "cf0b558b-4f01-49d5-97fc-419e9218340b") : secret "maas-api-e2e-api-a-cd3c29-serving-cert" not found
Sep 11 13:19:07.072969 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:07.072943    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf0b558b-4f01-49d5-97fc-419e9218340b-openshift-service-ca\") pod \"maas-api-e2e-api-a-cd3c29-5d96779975-55mdg\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:07.074657 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:07.074632    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf0b558b-4f01-49d5-97fc-419e9218340b-metrics-certs\") pod \"maas-api-e2e-api-a-cd3c29-5d96779975-55mdg\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:07.087399 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:07.087351    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxftq\" (UniqueName: \"kubernetes.io/projected/cf0b558b-4f01-49d5-97fc-419e9218340b-kube-api-access-bxftq\") pod \"maas-api-e2e-api-a-cd3c29-5d96779975-55mdg\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:07.577021 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:07.576983    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf0b558b-4f01-49d5-97fc-419e9218340b-maas-api-tls\") pod \"maas-api-e2e-api-a-cd3c29-5d96779975-55mdg\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:07.579298 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:07.579273    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf0b558b-4f01-49d5-97fc-419e9218340b-maas-api-tls\") pod \"maas-api-e2e-api-a-cd3c29-5d96779975-55mdg\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:07.752431 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:07.752385    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:08.031756 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:08.031728    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"]
Sep 11 13:19:08.033073 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:19:08.033045    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf0b558b_4f01_49d5_97fc_419e9218340b.slice/crio-8eace1325c78992554b8f1aff51a1ed7ce52d777645e1b6a3903d99d03016e50 WatchSource:0}: Error finding container 8eace1325c78992554b8f1aff51a1ed7ce52d777645e1b6a3903d99d03016e50: Status 404 returned error can't find the container with id 8eace1325c78992554b8f1aff51a1ed7ce52d777645e1b6a3903d99d03016e50
Sep 11 13:19:08.034592 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:08.034572    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:19:08.895576 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:08.895529    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg" event={"ID":"cf0b558b-4f01-49d5-97fc-419e9218340b","Type":"ContainerStarted","Data":"630bdde384a4f991647352dbd3996aa1a2dcc70a0d549c393bea635518e9d344"}
Sep 11 13:19:08.895759 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:08.895594    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:08.895759 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:08.895610    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg" event={"ID":"cf0b558b-4f01-49d5-97fc-419e9218340b","Type":"ContainerStarted","Data":"8eace1325c78992554b8f1aff51a1ed7ce52d777645e1b6a3903d99d03016e50"}
Sep 11 13:19:08.915700 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:08.915656    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg" podStartSLOduration=2.531227811 podStartE2EDuration="2.915640628s" totalImagesPullingTime="384.412817ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:08.913888284 +0000 UTC m=+1402.853589641" watchObservedRunningTime="2026-09-11 13:19:08.915640628 +0000 UTC m=+1402.855341961"
Sep 11 13:19:14.906748 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:14.906720    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:19:36.712351 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.712315    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"]
Sep 11 13:19:36.716058 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.716038    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:19:36.718875 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.718854    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a3785d-serving-cert\""
Sep 11 13:19:36.729459 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.729434    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"]
Sep 11 13:19:36.835053 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.835013    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e51a5476-5640-4f26-b95d-8d7ae0afd019-metrics-certs\") pod \"maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:19:36.835053 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.835062    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e51a5476-5640-4f26-b95d-8d7ae0afd019-maas-api-tls\") pod \"maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:19:36.835315 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.835088    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e51a5476-5640-4f26-b95d-8d7ae0afd019-openshift-service-ca\") pod \"maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:19:36.835315 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.835152    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvngv\" (UniqueName: \"kubernetes.io/projected/e51a5476-5640-4f26-b95d-8d7ae0afd019-kube-api-access-nvngv\") pod \"maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:19:36.936241 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.936210    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e51a5476-5640-4f26-b95d-8d7ae0afd019-metrics-certs\") pod \"maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:19:36.936416 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.936256    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e51a5476-5640-4f26-b95d-8d7ae0afd019-maas-api-tls\") pod \"maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:19:36.936416 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.936299    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e51a5476-5640-4f26-b95d-8d7ae0afd019-openshift-service-ca\") pod \"maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:19:36.936416 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.936375    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvngv\" (UniqueName: \"kubernetes.io/projected/e51a5476-5640-4f26-b95d-8d7ae0afd019-kube-api-access-nvngv\") pod \"maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:19:36.937055 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.937032    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e51a5476-5640-4f26-b95d-8d7ae0afd019-openshift-service-ca\") pod \"maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:19:36.938801 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.938784    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e51a5476-5640-4f26-b95d-8d7ae0afd019-metrics-certs\") pod \"maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:19:36.938884 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.938799    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e51a5476-5640-4f26-b95d-8d7ae0afd019-maas-api-tls\") pod \"maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:19:36.947048 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:36.947024    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvngv\" (UniqueName: \"kubernetes.io/projected/e51a5476-5640-4f26-b95d-8d7ae0afd019-kube-api-access-nvngv\") pod \"maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:19:37.028726 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:37.028684    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:19:37.313856 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:37.313830    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"]
Sep 11 13:19:37.316137 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:19:37.316110    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode51a5476_5640_4f26_b95d_8d7ae0afd019.slice/crio-8b072fb8026c4bee1a5bd2f379bbb73b9c5aa382bbd2ef26f651152f49847556 WatchSource:0}: Error finding container 8b072fb8026c4bee1a5bd2f379bbb73b9c5aa382bbd2ef26f651152f49847556: Status 404 returned error can't find the container with id 8b072fb8026c4bee1a5bd2f379bbb73b9c5aa382bbd2ef26f651152f49847556
Sep 11 13:19:38.053035 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:38.052984    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb" event={"ID":"e51a5476-5640-4f26-b95d-8d7ae0afd019","Type":"ContainerStarted","Data":"9decf21b763f6d5d0d82e27abfc0832545c78af1fc087307c097c4391f0103ae"}
Sep 11 13:19:38.053519 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:38.053107    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:19:38.053519 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:38.053134    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb" event={"ID":"e51a5476-5640-4f26-b95d-8d7ae0afd019","Type":"ContainerStarted","Data":"8b072fb8026c4bee1a5bd2f379bbb73b9c5aa382bbd2ef26f651152f49847556"}
Sep 11 13:19:38.085158 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:38.085109    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb" podStartSLOduration=1.798164146 podStartE2EDuration="2.085093667s" totalImagesPullingTime="286.929521ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:38.084065709 +0000 UTC m=+1432.023767042" watchObservedRunningTime="2026-09-11 13:19:38.085093667 +0000 UTC m=+1432.024795000"
Sep 11 13:19:44.065369 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:19:44.065293    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:20:18.149920 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.149880    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"]
Sep 11 13:20:18.153929 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.153895    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:18.157073 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.157046    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-cc857f-serving-cert\""
Sep 11 13:20:18.171860 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.171825    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"]
Sep 11 13:20:18.251968 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.251927    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-cc857f-c49678896-rd78q\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:18.252156 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.251977    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr4qn\" (UniqueName: \"kubernetes.io/projected/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-kube-api-access-sr4qn\") pod \"maas-api-e2e-ipp-a-cc857f-c49678896-rd78q\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:18.252156 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.252105    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-metrics-certs\") pod \"maas-api-e2e-ipp-a-cc857f-c49678896-rd78q\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:18.252244 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.252156    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-cc857f-c49678896-rd78q\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:18.353733 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.353699    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-cc857f-c49678896-rd78q\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:18.353733 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.353747    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr4qn\" (UniqueName: \"kubernetes.io/projected/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-kube-api-access-sr4qn\") pod \"maas-api-e2e-ipp-a-cc857f-c49678896-rd78q\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:18.354003 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.353789    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-metrics-certs\") pod \"maas-api-e2e-ipp-a-cc857f-c49678896-rd78q\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:18.354003 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.353814    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-cc857f-c49678896-rd78q\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:18.354003 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:20:18.353947    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-serving-cert: secret "maas-api-e2e-ipp-a-cc857f-serving-cert" not found
Sep 11 13:20:18.354172 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:20:18.354023    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-maas-api-tls podName:dd4bd540-e78e-4939-b8f6-5ca882c3a93c nodeName:}" failed. No retries permitted until 2026-09-11 13:20:18.854002599 +0000 UTC m=+1472.793703950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-maas-api-tls") pod "maas-api-e2e-ipp-a-cc857f-c49678896-rd78q" (UID: "dd4bd540-e78e-4939-b8f6-5ca882c3a93c") : secret "maas-api-e2e-ipp-a-cc857f-serving-cert" not found
Sep 11 13:20:18.354538 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.354513    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-cc857f-c49678896-rd78q\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:18.356335 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.356313    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-metrics-certs\") pod \"maas-api-e2e-ipp-a-cc857f-c49678896-rd78q\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:18.364303 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.364277    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr4qn\" (UniqueName: \"kubernetes.io/projected/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-kube-api-access-sr4qn\") pod \"maas-api-e2e-ipp-a-cc857f-c49678896-rd78q\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:18.860049 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.860008    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-cc857f-c49678896-rd78q\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:18.862729 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:18.862698    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-cc857f-c49678896-rd78q\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:19.067124 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:19.067089    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:19.351755 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:19.351709    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"]
Sep 11 13:20:19.353879 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:20:19.353849    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd4bd540_e78e_4939_b8f6_5ca882c3a93c.slice/crio-7fb0250967095a1c8d810a124782778ed38321b8415359a3460a608932061dc3 WatchSource:0}: Error finding container 7fb0250967095a1c8d810a124782778ed38321b8415359a3460a608932061dc3: Status 404 returned error can't find the container with id 7fb0250967095a1c8d810a124782778ed38321b8415359a3460a608932061dc3
Sep 11 13:20:20.294470 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:20.294407    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q" event={"ID":"dd4bd540-e78e-4939-b8f6-5ca882c3a93c","Type":"ContainerStarted","Data":"2e3d99e826af9a34d3b5f804dddde6bbfdf091fa6cb1bb7f5bf01ec560a57e9b"}
Sep 11 13:20:20.294659 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:20.294498    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q" event={"ID":"dd4bd540-e78e-4939-b8f6-5ca882c3a93c","Type":"ContainerStarted","Data":"7fb0250967095a1c8d810a124782778ed38321b8415359a3460a608932061dc3"}
Sep 11 13:20:20.294659 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:20.294528    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:20.347016 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:20.346946    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q" podStartSLOduration=1.967498433 podStartE2EDuration="2.346928771s" totalImagesPullingTime="379.430338ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:20:20.34499962 +0000 UTC m=+1474.284700952" watchObservedRunningTime="2026-09-11 13:20:20.346928771 +0000 UTC m=+1474.286630103"
Sep 11 13:20:21.135675 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.135635    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"]
Sep 11 13:20:21.136047 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.135854    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg" podUID="cf0b558b-4f01-49d5-97fc-419e9218340b" containerName="maas-api" containerID="cri-o://630bdde384a4f991647352dbd3996aa1a2dcc70a0d549c393bea635518e9d344" gracePeriod=30
Sep 11 13:20:21.302525 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.302492    2582 generic.go:356] "Generic (PLEG): container finished" podID="cf0b558b-4f01-49d5-97fc-419e9218340b" containerID="630bdde384a4f991647352dbd3996aa1a2dcc70a0d549c393bea635518e9d344" exitCode=0
Sep 11 13:20:21.302683 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.302557    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg" event={"ID":"cf0b558b-4f01-49d5-97fc-419e9218340b","Type":"ContainerDied","Data":"630bdde384a4f991647352dbd3996aa1a2dcc70a0d549c393bea635518e9d344"}
Sep 11 13:20:21.409917 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.409890    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:20:21.488215 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.488180    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf0b558b-4f01-49d5-97fc-419e9218340b-openshift-service-ca\") pod \"cf0b558b-4f01-49d5-97fc-419e9218340b\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") "
Sep 11 13:20:21.488388 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.488267    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxftq\" (UniqueName: \"kubernetes.io/projected/cf0b558b-4f01-49d5-97fc-419e9218340b-kube-api-access-bxftq\") pod \"cf0b558b-4f01-49d5-97fc-419e9218340b\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") "
Sep 11 13:20:21.488388 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.488314    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf0b558b-4f01-49d5-97fc-419e9218340b-metrics-certs\") pod \"cf0b558b-4f01-49d5-97fc-419e9218340b\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") "
Sep 11 13:20:21.488388 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.488379    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf0b558b-4f01-49d5-97fc-419e9218340b-maas-api-tls\") pod \"cf0b558b-4f01-49d5-97fc-419e9218340b\" (UID: \"cf0b558b-4f01-49d5-97fc-419e9218340b\") "
Sep 11 13:20:21.488677 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.488647    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf0b558b-4f01-49d5-97fc-419e9218340b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cf0b558b-4f01-49d5-97fc-419e9218340b" (UID: "cf0b558b-4f01-49d5-97fc-419e9218340b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:20:21.490630 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.490601    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf0b558b-4f01-49d5-97fc-419e9218340b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cf0b558b-4f01-49d5-97fc-419e9218340b" (UID: "cf0b558b-4f01-49d5-97fc-419e9218340b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:21.490755 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.490610    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf0b558b-4f01-49d5-97fc-419e9218340b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cf0b558b-4f01-49d5-97fc-419e9218340b" (UID: "cf0b558b-4f01-49d5-97fc-419e9218340b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:21.490755 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.490674    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf0b558b-4f01-49d5-97fc-419e9218340b-kube-api-access-bxftq" (OuterVolumeSpecName: "kube-api-access-bxftq") pod "cf0b558b-4f01-49d5-97fc-419e9218340b" (UID: "cf0b558b-4f01-49d5-97fc-419e9218340b"). InnerVolumeSpecName "kube-api-access-bxftq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:20:21.589800 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.589763    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf0b558b-4f01-49d5-97fc-419e9218340b-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:20:21.589800 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.589800    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf0b558b-4f01-49d5-97fc-419e9218340b-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:20:21.589997 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.589815    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxftq\" (UniqueName: \"kubernetes.io/projected/cf0b558b-4f01-49d5-97fc-419e9218340b-kube-api-access-bxftq\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:20:21.589997 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:21.589828    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf0b558b-4f01-49d5-97fc-419e9218340b-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:20:22.309332 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:22.309301    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"
Sep 11 13:20:22.309812 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:22.309332    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg" event={"ID":"cf0b558b-4f01-49d5-97fc-419e9218340b","Type":"ContainerDied","Data":"8eace1325c78992554b8f1aff51a1ed7ce52d777645e1b6a3903d99d03016e50"}
Sep 11 13:20:22.309812 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:22.309403    2582 scope.go:117] "RemoveContainer" containerID="630bdde384a4f991647352dbd3996aa1a2dcc70a0d549c393bea635518e9d344"
Sep 11 13:20:22.365779 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:22.365747    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"]
Sep 11 13:20:22.379549 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:22.379527    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cd3c29-5d96779975-55mdg"]
Sep 11 13:20:22.477768 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:22.477732    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf0b558b-4f01-49d5-97fc-419e9218340b" path="/var/lib/kubelet/pods/cf0b558b-4f01-49d5-97fc-419e9218340b/volumes"
Sep 11 13:20:26.309076 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:26.309041    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:20:43.180816 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.180775    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"]
Sep 11 13:20:43.181336 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.181321    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf0b558b-4f01-49d5-97fc-419e9218340b" containerName="maas-api"
Sep 11 13:20:43.181377 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.181338    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf0b558b-4f01-49d5-97fc-419e9218340b" containerName="maas-api"
Sep 11 13:20:43.181459 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.181445    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf0b558b-4f01-49d5-97fc-419e9218340b" containerName="maas-api"
Sep 11 13:20:43.186677 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.186654    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:43.192293 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.192263    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-e3f42f-serving-cert\""
Sep 11 13:20:43.221543 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.221508    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"]
Sep 11 13:20:43.316201 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.316152    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x92jh\" (UniqueName: \"kubernetes.io/projected/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-kube-api-access-x92jh\") pod \"maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:43.316393 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.316211    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:43.316393 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.316271    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:43.316393 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.316332    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-metrics-certs\") pod \"maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:43.417323 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.417288    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x92jh\" (UniqueName: \"kubernetes.io/projected/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-kube-api-access-x92jh\") pod \"maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:43.417586 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.417334    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:43.417586 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.417366    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:43.417586 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.417413    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-metrics-certs\") pod \"maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:43.417586 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:20:43.417531    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-serving-cert: secret "maas-api-e2e-ipp-b-e3f42f-serving-cert" not found
Sep 11 13:20:43.417830 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:20:43.417621    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-maas-api-tls podName:4ecd8b7f-48b1-40f4-b39e-687b5c0958c8 nodeName:}" failed. No retries permitted until 2026-09-11 13:20:43.917598679 +0000 UTC m=+1497.857300010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-maas-api-tls") pod "maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g" (UID: "4ecd8b7f-48b1-40f4-b39e-687b5c0958c8") : secret "maas-api-e2e-ipp-b-e3f42f-serving-cert" not found
Sep 11 13:20:43.418169 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.418147    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:43.419886 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.419858    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-metrics-certs\") pod \"maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:43.428365 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.428334    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x92jh\" (UniqueName: \"kubernetes.io/projected/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-kube-api-access-x92jh\") pod \"maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:43.922190 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.922153    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:43.924853 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:43.924825    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:44.097584 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:44.097543    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:44.433356 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:44.433325    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"]
Sep 11 13:20:44.436086 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:20:44.436051    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ecd8b7f_48b1_40f4_b39e_687b5c0958c8.slice/crio-8ec850f78001acd4b844fae3cbab6dbf87e0cc6f9215485184c3ab70ab83f070 WatchSource:0}: Error finding container 8ec850f78001acd4b844fae3cbab6dbf87e0cc6f9215485184c3ab70ab83f070: Status 404 returned error can't find the container with id 8ec850f78001acd4b844fae3cbab6dbf87e0cc6f9215485184c3ab70ab83f070
Sep 11 13:20:44.447303 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:44.447261    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g" event={"ID":"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8","Type":"ContainerStarted","Data":"8ec850f78001acd4b844fae3cbab6dbf87e0cc6f9215485184c3ab70ab83f070"}
Sep 11 13:20:45.456068 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:45.456010    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g" event={"ID":"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8","Type":"ContainerStarted","Data":"90c6317460364a9c3f130effc2e8a8664bebb931480606b79098f2b385cc2bdd"}
Sep 11 13:20:45.456590 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:45.456089    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:45.506973 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:45.506915    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g" podStartSLOduration=2.076604826 podStartE2EDuration="2.50689561s" totalImagesPullingTime="430.290784ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:20:45.503966717 +0000 UTC m=+1499.443668063" watchObservedRunningTime="2026-09-11 13:20:45.50689561 +0000 UTC m=+1499.446596944"
Sep 11 13:20:51.467296 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:51.467263    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:20:51.839372 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:51.839338    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"]
Sep 11 13:20:51.839597 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:51.839548    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb" podUID="e51a5476-5640-4f26-b95d-8d7ae0afd019" containerName="maas-api" containerID="cri-o://9decf21b763f6d5d0d82e27abfc0832545c78af1fc087307c097c4391f0103ae" gracePeriod=30
Sep 11 13:20:52.117434 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.117392    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:20:52.202758 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.202723    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e51a5476-5640-4f26-b95d-8d7ae0afd019-openshift-service-ca\") pod \"e51a5476-5640-4f26-b95d-8d7ae0afd019\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") "
Sep 11 13:20:52.202928 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.202786    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvngv\" (UniqueName: \"kubernetes.io/projected/e51a5476-5640-4f26-b95d-8d7ae0afd019-kube-api-access-nvngv\") pod \"e51a5476-5640-4f26-b95d-8d7ae0afd019\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") "
Sep 11 13:20:52.202928 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.202804    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e51a5476-5640-4f26-b95d-8d7ae0afd019-maas-api-tls\") pod \"e51a5476-5640-4f26-b95d-8d7ae0afd019\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") "
Sep 11 13:20:52.202928 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.202877    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e51a5476-5640-4f26-b95d-8d7ae0afd019-metrics-certs\") pod \"e51a5476-5640-4f26-b95d-8d7ae0afd019\" (UID: \"e51a5476-5640-4f26-b95d-8d7ae0afd019\") "
Sep 11 13:20:52.203256 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.203200    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e51a5476-5640-4f26-b95d-8d7ae0afd019-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e51a5476-5640-4f26-b95d-8d7ae0afd019" (UID: "e51a5476-5640-4f26-b95d-8d7ae0afd019"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:20:52.205243 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.205210    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e51a5476-5640-4f26-b95d-8d7ae0afd019-kube-api-access-nvngv" (OuterVolumeSpecName: "kube-api-access-nvngv") pod "e51a5476-5640-4f26-b95d-8d7ae0afd019" (UID: "e51a5476-5640-4f26-b95d-8d7ae0afd019"). InnerVolumeSpecName "kube-api-access-nvngv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:20:52.205346 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.205210    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e51a5476-5640-4f26-b95d-8d7ae0afd019-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e51a5476-5640-4f26-b95d-8d7ae0afd019" (UID: "e51a5476-5640-4f26-b95d-8d7ae0afd019"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:52.205346 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.205242    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e51a5476-5640-4f26-b95d-8d7ae0afd019-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e51a5476-5640-4f26-b95d-8d7ae0afd019" (UID: "e51a5476-5640-4f26-b95d-8d7ae0afd019"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:20:52.304571 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.304527    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e51a5476-5640-4f26-b95d-8d7ae0afd019-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:20:52.304748 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.304561    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvngv\" (UniqueName: \"kubernetes.io/projected/e51a5476-5640-4f26-b95d-8d7ae0afd019-kube-api-access-nvngv\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:20:52.304748 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.304596    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e51a5476-5640-4f26-b95d-8d7ae0afd019-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:20:52.304748 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.304608    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e51a5476-5640-4f26-b95d-8d7ae0afd019-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:20:52.495627 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.495596    2582 generic.go:356] "Generic (PLEG): container finished" podID="e51a5476-5640-4f26-b95d-8d7ae0afd019" containerID="9decf21b763f6d5d0d82e27abfc0832545c78af1fc087307c097c4391f0103ae" exitCode=0
Sep 11 13:20:52.495979 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.495642    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb" event={"ID":"e51a5476-5640-4f26-b95d-8d7ae0afd019","Type":"ContainerDied","Data":"9decf21b763f6d5d0d82e27abfc0832545c78af1fc087307c097c4391f0103ae"}
Sep 11 13:20:52.495979 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.495685    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb" event={"ID":"e51a5476-5640-4f26-b95d-8d7ae0afd019","Type":"ContainerDied","Data":"8b072fb8026c4bee1a5bd2f379bbb73b9c5aa382bbd2ef26f651152f49847556"}
Sep 11 13:20:52.495979 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.495707    2582 scope.go:117] "RemoveContainer" containerID="9decf21b763f6d5d0d82e27abfc0832545c78af1fc087307c097c4391f0103ae"
Sep 11 13:20:52.495979 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.495644    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"
Sep 11 13:20:52.505320 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.505297    2582 scope.go:117] "RemoveContainer" containerID="9decf21b763f6d5d0d82e27abfc0832545c78af1fc087307c097c4391f0103ae"
Sep 11 13:20:52.505654 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:20:52.505621    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9decf21b763f6d5d0d82e27abfc0832545c78af1fc087307c097c4391f0103ae\": container with ID starting with 9decf21b763f6d5d0d82e27abfc0832545c78af1fc087307c097c4391f0103ae not found: ID does not exist" containerID="9decf21b763f6d5d0d82e27abfc0832545c78af1fc087307c097c4391f0103ae"
Sep 11 13:20:52.505756 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.505663    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9decf21b763f6d5d0d82e27abfc0832545c78af1fc087307c097c4391f0103ae"} err="failed to get container status \"9decf21b763f6d5d0d82e27abfc0832545c78af1fc087307c097c4391f0103ae\": rpc error: code = NotFound desc = could not find container \"9decf21b763f6d5d0d82e27abfc0832545c78af1fc087307c097c4391f0103ae\": container with ID starting with 9decf21b763f6d5d0d82e27abfc0832545c78af1fc087307c097c4391f0103ae not found: ID does not exist"
Sep 11 13:20:52.536994 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.536961    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"]
Sep 11 13:20:52.554154 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:52.554116    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a3785d-57bd9b99bd-gz4pb"]
Sep 11 13:20:54.478674 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:20:54.478632    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e51a5476-5640-4f26-b95d-8d7ae0afd019" path="/var/lib/kubelet/pods/e51a5476-5640-4f26-b95d-8d7ae0afd019/volumes"
Sep 11 13:21:05.757938 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:05.757893    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-84qbs_71f3fe4f-d604-49ce-8364-6643e5c1afc1/payload-processing/0.log"
Sep 11 13:21:12.811506 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:12.811412    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"]
Sep 11 13:21:12.811897 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:12.811655    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz" podUID="4dfc3b6e-172d-4d4e-82b1-1afbf65f3596" containerName="maas-api" containerID="cri-o://8035306a8128067cd88e25c3959e55b7b691fd28e7cbeac8e93a73d2ce88ad5c" gracePeriod=30
Sep 11 13:21:13.079128 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.079063    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:21:13.129829 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.129784    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-metrics-certs\") pod \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") "
Sep 11 13:21:13.130028 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.129868    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-maas-api-tls\") pod \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") "
Sep 11 13:21:13.130028 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.130023    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6vzr\" (UniqueName: \"kubernetes.io/projected/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-kube-api-access-x6vzr\") pod \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") "
Sep 11 13:21:13.130169 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.130136    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-openshift-service-ca\") pod \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\" (UID: \"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596\") "
Sep 11 13:21:13.131686 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.130827    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4dfc3b6e-172d-4d4e-82b1-1afbf65f3596" (UID: "4dfc3b6e-172d-4d4e-82b1-1afbf65f3596"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:21:13.132512 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.132487    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4dfc3b6e-172d-4d4e-82b1-1afbf65f3596" (UID: "4dfc3b6e-172d-4d4e-82b1-1afbf65f3596"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:21:13.132625 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.132525    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4dfc3b6e-172d-4d4e-82b1-1afbf65f3596" (UID: "4dfc3b6e-172d-4d4e-82b1-1afbf65f3596"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:21:13.132955 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.132939    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-kube-api-access-x6vzr" (OuterVolumeSpecName: "kube-api-access-x6vzr") pod "4dfc3b6e-172d-4d4e-82b1-1afbf65f3596" (UID: "4dfc3b6e-172d-4d4e-82b1-1afbf65f3596"). InnerVolumeSpecName "kube-api-access-x6vzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:21:13.231887 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.231851    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6vzr\" (UniqueName: \"kubernetes.io/projected/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-kube-api-access-x6vzr\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:21:13.231887 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.231881    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:21:13.231887 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.231892    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:21:13.232121 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.231901    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:21:13.619452 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.619398    2582 generic.go:356] "Generic (PLEG): container finished" podID="4dfc3b6e-172d-4d4e-82b1-1afbf65f3596" containerID="8035306a8128067cd88e25c3959e55b7b691fd28e7cbeac8e93a73d2ce88ad5c" exitCode=0
Sep 11 13:21:13.619629 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.619457    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz" event={"ID":"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596","Type":"ContainerDied","Data":"8035306a8128067cd88e25c3959e55b7b691fd28e7cbeac8e93a73d2ce88ad5c"}
Sep 11 13:21:13.619629 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.619500    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"
Sep 11 13:21:13.619629 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.619523    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz" event={"ID":"4dfc3b6e-172d-4d4e-82b1-1afbf65f3596","Type":"ContainerDied","Data":"a69bee986eda4b8f6ed27b7d4bd410bb49ba56c0c7361f822e47c68a05e5003f"}
Sep 11 13:21:13.619629 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.619549    2582 scope.go:117] "RemoveContainer" containerID="8035306a8128067cd88e25c3959e55b7b691fd28e7cbeac8e93a73d2ce88ad5c"
Sep 11 13:21:13.630556 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.630540    2582 scope.go:117] "RemoveContainer" containerID="8035306a8128067cd88e25c3959e55b7b691fd28e7cbeac8e93a73d2ce88ad5c"
Sep 11 13:21:13.630832 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:21:13.630814    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8035306a8128067cd88e25c3959e55b7b691fd28e7cbeac8e93a73d2ce88ad5c\": container with ID starting with 8035306a8128067cd88e25c3959e55b7b691fd28e7cbeac8e93a73d2ce88ad5c not found: ID does not exist" containerID="8035306a8128067cd88e25c3959e55b7b691fd28e7cbeac8e93a73d2ce88ad5c"
Sep 11 13:21:13.630875 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.630844    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8035306a8128067cd88e25c3959e55b7b691fd28e7cbeac8e93a73d2ce88ad5c"} err="failed to get container status \"8035306a8128067cd88e25c3959e55b7b691fd28e7cbeac8e93a73d2ce88ad5c\": rpc error: code = NotFound desc = could not find container \"8035306a8128067cd88e25c3959e55b7b691fd28e7cbeac8e93a73d2ce88ad5c\": container with ID starting with 8035306a8128067cd88e25c3959e55b7b691fd28e7cbeac8e93a73d2ce88ad5c not found: ID does not exist"
Sep 11 13:21:13.644549 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.644523    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"]
Sep 11 13:21:13.648267 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:13.648246    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-4d7bb0-689b7fbff-rvqvz"]
Sep 11 13:21:14.476358 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:14.476327    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dfc3b6e-172d-4d4e-82b1-1afbf65f3596" path="/var/lib/kubelet/pods/4dfc3b6e-172d-4d4e-82b1-1afbf65f3596/volumes"
Sep 11 13:21:33.462099 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.462056    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"]
Sep 11 13:21:33.462688 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.462297    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5" podUID="466bb846-b3d5-41a9-b52e-22466cb40b29" containerName="maas-api" containerID="cri-o://c77aaac89e4d1650a9fefbb2c8fc5ce98755069f378fceb3959e80c5cc53f8fe" gracePeriod=30
Sep 11 13:21:33.726723 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.726694    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:21:33.730445 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.730403    2582 generic.go:356] "Generic (PLEG): container finished" podID="466bb846-b3d5-41a9-b52e-22466cb40b29" containerID="c77aaac89e4d1650a9fefbb2c8fc5ce98755069f378fceb3959e80c5cc53f8fe" exitCode=0
Sep 11 13:21:33.730576 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.730452    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5" event={"ID":"466bb846-b3d5-41a9-b52e-22466cb40b29","Type":"ContainerDied","Data":"c77aaac89e4d1650a9fefbb2c8fc5ce98755069f378fceb3959e80c5cc53f8fe"}
Sep 11 13:21:33.730576 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.730474    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"
Sep 11 13:21:33.730576 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.730531    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5" event={"ID":"466bb846-b3d5-41a9-b52e-22466cb40b29","Type":"ContainerDied","Data":"5b9a0f17adbd006fd8a7d5c91e5278dc4b943dd2c84263a4c8481f52ce493a27"}
Sep 11 13:21:33.730576 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.730564    2582 scope.go:117] "RemoveContainer" containerID="c77aaac89e4d1650a9fefbb2c8fc5ce98755069f378fceb3959e80c5cc53f8fe"
Sep 11 13:21:33.740066 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.740049    2582 scope.go:117] "RemoveContainer" containerID="c77aaac89e4d1650a9fefbb2c8fc5ce98755069f378fceb3959e80c5cc53f8fe"
Sep 11 13:21:33.740361 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:21:33.740338    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c77aaac89e4d1650a9fefbb2c8fc5ce98755069f378fceb3959e80c5cc53f8fe\": container with ID starting with c77aaac89e4d1650a9fefbb2c8fc5ce98755069f378fceb3959e80c5cc53f8fe not found: ID does not exist" containerID="c77aaac89e4d1650a9fefbb2c8fc5ce98755069f378fceb3959e80c5cc53f8fe"
Sep 11 13:21:33.740483 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.740366    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c77aaac89e4d1650a9fefbb2c8fc5ce98755069f378fceb3959e80c5cc53f8fe"} err="failed to get container status \"c77aaac89e4d1650a9fefbb2c8fc5ce98755069f378fceb3959e80c5cc53f8fe\": rpc error: code = NotFound desc = could not find container \"c77aaac89e4d1650a9fefbb2c8fc5ce98755069f378fceb3959e80c5cc53f8fe\": container with ID starting with c77aaac89e4d1650a9fefbb2c8fc5ce98755069f378fceb3959e80c5cc53f8fe not found: ID does not exist"
Sep 11 13:21:33.850800 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.850767    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/466bb846-b3d5-41a9-b52e-22466cb40b29-openshift-service-ca\") pod \"466bb846-b3d5-41a9-b52e-22466cb40b29\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") "
Sep 11 13:21:33.850977 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.850814    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/466bb846-b3d5-41a9-b52e-22466cb40b29-metrics-certs\") pod \"466bb846-b3d5-41a9-b52e-22466cb40b29\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") "
Sep 11 13:21:33.850977 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.850909    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466bb846-b3d5-41a9-b52e-22466cb40b29-maas-api-tls\") pod \"466bb846-b3d5-41a9-b52e-22466cb40b29\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") "
Sep 11 13:21:33.851053 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.851010    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ldtm\" (UniqueName: \"kubernetes.io/projected/466bb846-b3d5-41a9-b52e-22466cb40b29-kube-api-access-4ldtm\") pod \"466bb846-b3d5-41a9-b52e-22466cb40b29\" (UID: \"466bb846-b3d5-41a9-b52e-22466cb40b29\") "
Sep 11 13:21:33.851212 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.851192    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/466bb846-b3d5-41a9-b52e-22466cb40b29-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "466bb846-b3d5-41a9-b52e-22466cb40b29" (UID: "466bb846-b3d5-41a9-b52e-22466cb40b29"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:21:33.851841 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.851411    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/466bb846-b3d5-41a9-b52e-22466cb40b29-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:21:33.854508 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.854396    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/466bb846-b3d5-41a9-b52e-22466cb40b29-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "466bb846-b3d5-41a9-b52e-22466cb40b29" (UID: "466bb846-b3d5-41a9-b52e-22466cb40b29"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:21:33.854508 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.854471    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/466bb846-b3d5-41a9-b52e-22466cb40b29-kube-api-access-4ldtm" (OuterVolumeSpecName: "kube-api-access-4ldtm") pod "466bb846-b3d5-41a9-b52e-22466cb40b29" (UID: "466bb846-b3d5-41a9-b52e-22466cb40b29"). InnerVolumeSpecName "kube-api-access-4ldtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:21:33.855295 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.855262    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/466bb846-b3d5-41a9-b52e-22466cb40b29-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "466bb846-b3d5-41a9-b52e-22466cb40b29" (UID: "466bb846-b3d5-41a9-b52e-22466cb40b29"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:21:33.952100 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.952065    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466bb846-b3d5-41a9-b52e-22466cb40b29-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:21:33.952100 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.952095    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ldtm\" (UniqueName: \"kubernetes.io/projected/466bb846-b3d5-41a9-b52e-22466cb40b29-kube-api-access-4ldtm\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:21:33.952100 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:33.952105    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/466bb846-b3d5-41a9-b52e-22466cb40b29-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:21:34.076447 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:34.076392    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"]
Sep 11 13:21:34.078692 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:34.078669    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f705b6-5886885c-cdwl5"]
Sep 11 13:21:34.476869 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:34.476837    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="466bb846-b3d5-41a9-b52e-22466cb40b29" path="/var/lib/kubelet/pods/466bb846-b3d5-41a9-b52e-22466cb40b29/volumes"
Sep 11 13:21:52.259438 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:21:52.259398    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-84qbs_71f3fe4f-d604-49ce-8364-6643e5c1afc1/payload-processing/0.log"
Sep 11 13:22:01.828254 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.828221    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"]
Sep 11 13:22:01.828759 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.828730    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="466bb846-b3d5-41a9-b52e-22466cb40b29" containerName="maas-api"
Sep 11 13:22:01.828759 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.828744    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="466bb846-b3d5-41a9-b52e-22466cb40b29" containerName="maas-api"
Sep 11 13:22:01.828759 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.828754    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e51a5476-5640-4f26-b95d-8d7ae0afd019" containerName="maas-api"
Sep 11 13:22:01.828759 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.828759    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e51a5476-5640-4f26-b95d-8d7ae0afd019" containerName="maas-api"
Sep 11 13:22:01.828898 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.828780    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dfc3b6e-172d-4d4e-82b1-1afbf65f3596" containerName="maas-api"
Sep 11 13:22:01.828898 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.828786    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dfc3b6e-172d-4d4e-82b1-1afbf65f3596" containerName="maas-api"
Sep 11 13:22:01.828898 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.828856    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dfc3b6e-172d-4d4e-82b1-1afbf65f3596" containerName="maas-api"
Sep 11 13:22:01.828898 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.828869    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e51a5476-5640-4f26-b95d-8d7ae0afd019" containerName="maas-api"
Sep 11 13:22:01.828898 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.828877    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="466bb846-b3d5-41a9-b52e-22466cb40b29" containerName="maas-api"
Sep 11 13:22:01.832105 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.832087    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:01.836076 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:22:01.836043    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw\" is forbidden: User \"system:node:ip-10-0-143-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-143-6.ec2.internal' and this object" podUID="e45ab133-d279-41a5-8ee6-2ee43c987a58" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:01.836195 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:22:01.836146    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-e2e-ipp-cleanup-48a080-serving-cert\" is forbidden: User \"system:node:ip-10-0-143-6.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-143-6.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-48a080-serving-cert\"" type="*v1.Secret"
Sep 11 13:22:01.865510 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.865478    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"]
Sep 11 13:22:01.952677 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.952645    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45ab133-d279-41a5-8ee6-2ee43c987a58-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:01.952837 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.952705    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbsgv\" (UniqueName: \"kubernetes.io/projected/e45ab133-d279-41a5-8ee6-2ee43c987a58-kube-api-access-cbsgv\") pod \"maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:01.952837 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.952764    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45ab133-d279-41a5-8ee6-2ee43c987a58-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:01.952837 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:01.952810    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45ab133-d279-41a5-8ee6-2ee43c987a58-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:02.053486 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:02.053443    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45ab133-d279-41a5-8ee6-2ee43c987a58-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:02.053665 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:02.053525    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbsgv\" (UniqueName: \"kubernetes.io/projected/e45ab133-d279-41a5-8ee6-2ee43c987a58-kube-api-access-cbsgv\") pod \"maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:02.053665 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:02.053589    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45ab133-d279-41a5-8ee6-2ee43c987a58-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:02.053665 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:02.053619    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45ab133-d279-41a5-8ee6-2ee43c987a58-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:02.054236 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:02.054211    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45ab133-d279-41a5-8ee6-2ee43c987a58-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:02.056122 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:02.056093    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45ab133-d279-41a5-8ee6-2ee43c987a58-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:02.067481 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:02.067456    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbsgv\" (UniqueName: \"kubernetes.io/projected/e45ab133-d279-41a5-8ee6-2ee43c987a58-kube-api-access-cbsgv\") pod \"maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:03.053993 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:22:03.053950    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-serving-cert: failed to sync secret cache: timed out waiting for the condition
Sep 11 13:22:03.054479 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:22:03.054075    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e45ab133-d279-41a5-8ee6-2ee43c987a58-maas-api-tls podName:e45ab133-d279-41a5-8ee6-2ee43c987a58 nodeName:}" failed. No retries permitted until 2026-09-11 13:22:03.554052017 +0000 UTC m=+1577.493753349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e45ab133-d279-41a5-8ee6-2ee43c987a58-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw" (UID: "e45ab133-d279-41a5-8ee6-2ee43c987a58") : failed to sync secret cache: timed out waiting for the condition
Sep 11 13:22:03.090592 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:03.090560    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-48a080-serving-cert\""
Sep 11 13:22:03.569392 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:03.569355    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45ab133-d279-41a5-8ee6-2ee43c987a58-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:03.571876 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:03.571842    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45ab133-d279-41a5-8ee6-2ee43c987a58-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:03.643972 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:03.643934    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:03.961444 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:03.961290    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"]
Sep 11 13:22:03.962699 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:22:03.962671    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45ab133_d279_41a5_8ee6_2ee43c987a58.slice/crio-dd60c06dfbbd14fb96f01a1e95244f027a444227b0e901038b7943a2313e2b00 WatchSource:0}: Error finding container dd60c06dfbbd14fb96f01a1e95244f027a444227b0e901038b7943a2313e2b00: Status 404 returned error can't find the container with id dd60c06dfbbd14fb96f01a1e95244f027a444227b0e901038b7943a2313e2b00
Sep 11 13:22:04.926194 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:04.926158    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw" event={"ID":"e45ab133-d279-41a5-8ee6-2ee43c987a58","Type":"ContainerStarted","Data":"dd60c06dfbbd14fb96f01a1e95244f027a444227b0e901038b7943a2313e2b00"}
Sep 11 13:22:05.933854 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:05.933818    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw" event={"ID":"e45ab133-d279-41a5-8ee6-2ee43c987a58","Type":"ContainerStarted","Data":"f295a8edc4084d5a136dd80c6420270af743699e296603a70180afafea411df6"}
Sep 11 13:22:05.934239 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:05.933942    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:05.973336 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:05.973286    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw" podStartSLOduration=3.934998868 podStartE2EDuration="4.973271809s" totalImagesPullingTime="1.038272941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:05.970067052 +0000 UTC m=+1579.909768384" watchObservedRunningTime="2026-09-11 13:22:05.973271809 +0000 UTC m=+1579.912973142"
Sep 11 13:22:11.944942 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:11.944904    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:38.611907 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:38.611870    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"]
Sep 11 13:22:38.612312 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:38.612077    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw" podUID="e45ab133-d279-41a5-8ee6-2ee43c987a58" containerName="maas-api" containerID="cri-o://f295a8edc4084d5a136dd80c6420270af743699e296603a70180afafea411df6" gracePeriod=30
Sep 11 13:22:38.883105 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:38.883078    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:39.008033 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.008002    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45ab133-d279-41a5-8ee6-2ee43c987a58-openshift-service-ca\") pod \"e45ab133-d279-41a5-8ee6-2ee43c987a58\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") "
Sep 11 13:22:39.008200 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.008060    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45ab133-d279-41a5-8ee6-2ee43c987a58-metrics-certs\") pod \"e45ab133-d279-41a5-8ee6-2ee43c987a58\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") "
Sep 11 13:22:39.008200 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.008106    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbsgv\" (UniqueName: \"kubernetes.io/projected/e45ab133-d279-41a5-8ee6-2ee43c987a58-kube-api-access-cbsgv\") pod \"e45ab133-d279-41a5-8ee6-2ee43c987a58\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") "
Sep 11 13:22:39.008286 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.008205    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45ab133-d279-41a5-8ee6-2ee43c987a58-maas-api-tls\") pod \"e45ab133-d279-41a5-8ee6-2ee43c987a58\" (UID: \"e45ab133-d279-41a5-8ee6-2ee43c987a58\") "
Sep 11 13:22:39.008469 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.008412    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e45ab133-d279-41a5-8ee6-2ee43c987a58-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e45ab133-d279-41a5-8ee6-2ee43c987a58" (UID: "e45ab133-d279-41a5-8ee6-2ee43c987a58"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:22:39.008638 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.008619    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45ab133-d279-41a5-8ee6-2ee43c987a58-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:22:39.010196 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.010173    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e45ab133-d279-41a5-8ee6-2ee43c987a58-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e45ab133-d279-41a5-8ee6-2ee43c987a58" (UID: "e45ab133-d279-41a5-8ee6-2ee43c987a58"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:22:39.010305 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.010290    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e45ab133-d279-41a5-8ee6-2ee43c987a58-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e45ab133-d279-41a5-8ee6-2ee43c987a58" (UID: "e45ab133-d279-41a5-8ee6-2ee43c987a58"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:22:39.010355 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.010334    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e45ab133-d279-41a5-8ee6-2ee43c987a58-kube-api-access-cbsgv" (OuterVolumeSpecName: "kube-api-access-cbsgv") pod "e45ab133-d279-41a5-8ee6-2ee43c987a58" (UID: "e45ab133-d279-41a5-8ee6-2ee43c987a58"). InnerVolumeSpecName "kube-api-access-cbsgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:22:39.109269 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.109235    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45ab133-d279-41a5-8ee6-2ee43c987a58-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:22:39.109269 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.109266    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbsgv\" (UniqueName: \"kubernetes.io/projected/e45ab133-d279-41a5-8ee6-2ee43c987a58-kube-api-access-cbsgv\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:22:39.109526 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.109281    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45ab133-d279-41a5-8ee6-2ee43c987a58-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:22:39.109670 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.109646    2582 generic.go:356] "Generic (PLEG): container finished" podID="e45ab133-d279-41a5-8ee6-2ee43c987a58" containerID="f295a8edc4084d5a136dd80c6420270af743699e296603a70180afafea411df6" exitCode=0
Sep 11 13:22:39.109745 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.109701    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"
Sep 11 13:22:39.109745 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.109725    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw" event={"ID":"e45ab133-d279-41a5-8ee6-2ee43c987a58","Type":"ContainerDied","Data":"f295a8edc4084d5a136dd80c6420270af743699e296603a70180afafea411df6"}
Sep 11 13:22:39.109826 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.109778    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw" event={"ID":"e45ab133-d279-41a5-8ee6-2ee43c987a58","Type":"ContainerDied","Data":"dd60c06dfbbd14fb96f01a1e95244f027a444227b0e901038b7943a2313e2b00"}
Sep 11 13:22:39.109826 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.109802    2582 scope.go:117] "RemoveContainer" containerID="f295a8edc4084d5a136dd80c6420270af743699e296603a70180afafea411df6"
Sep 11 13:22:39.120706 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.120691    2582 scope.go:117] "RemoveContainer" containerID="f295a8edc4084d5a136dd80c6420270af743699e296603a70180afafea411df6"
Sep 11 13:22:39.120943 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:22:39.120923    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f295a8edc4084d5a136dd80c6420270af743699e296603a70180afafea411df6\": container with ID starting with f295a8edc4084d5a136dd80c6420270af743699e296603a70180afafea411df6 not found: ID does not exist" containerID="f295a8edc4084d5a136dd80c6420270af743699e296603a70180afafea411df6"
Sep 11 13:22:39.121007 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.120955    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f295a8edc4084d5a136dd80c6420270af743699e296603a70180afafea411df6"} err="failed to get container status \"f295a8edc4084d5a136dd80c6420270af743699e296603a70180afafea411df6\": rpc error: code = NotFound desc = could not find container \"f295a8edc4084d5a136dd80c6420270af743699e296603a70180afafea411df6\": container with ID starting with f295a8edc4084d5a136dd80c6420270af743699e296603a70180afafea411df6 not found: ID does not exist"
Sep 11 13:22:39.135290 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.135268    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"]
Sep 11 13:22:39.146947 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:39.146924    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-48a080-64dd7b6c88-c98fw"]
Sep 11 13:22:40.481840 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:22:40.481802    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e45ab133-d279-41a5-8ee6-2ee43c987a58" path="/var/lib/kubelet/pods/e45ab133-d279-41a5-8ee6-2ee43c987a58/volumes"
Sep 11 13:23:05.570254 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:05.570213    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"]
Sep 11 13:23:05.570702 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:05.570407    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q" podUID="dd4bd540-e78e-4939-b8f6-5ca882c3a93c" containerName="maas-api" containerID="cri-o://2e3d99e826af9a34d3b5f804dddde6bbfdf091fa6cb1bb7f5bf01ec560a57e9b" gracePeriod=30
Sep 11 13:23:05.837990 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:05.837963    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:23:05.980639 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:05.980606    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-metrics-certs\") pod \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") "
Sep 11 13:23:05.980837 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:05.980697    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-openshift-service-ca\") pod \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") "
Sep 11 13:23:05.980837 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:05.980728    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sr4qn\" (UniqueName: \"kubernetes.io/projected/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-kube-api-access-sr4qn\") pod \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") "
Sep 11 13:23:05.980837 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:05.980782    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-maas-api-tls\") pod \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\" (UID: \"dd4bd540-e78e-4939-b8f6-5ca882c3a93c\") "
Sep 11 13:23:05.981359 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:05.981053    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dd4bd540-e78e-4939-b8f6-5ca882c3a93c" (UID: "dd4bd540-e78e-4939-b8f6-5ca882c3a93c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:23:05.981359 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:05.981183    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:23:05.983525 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:05.983496    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "dd4bd540-e78e-4939-b8f6-5ca882c3a93c" (UID: "dd4bd540-e78e-4939-b8f6-5ca882c3a93c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:23:05.983630 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:05.983519    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dd4bd540-e78e-4939-b8f6-5ca882c3a93c" (UID: "dd4bd540-e78e-4939-b8f6-5ca882c3a93c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:23:05.983630 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:05.983498    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-kube-api-access-sr4qn" (OuterVolumeSpecName: "kube-api-access-sr4qn") pod "dd4bd540-e78e-4939-b8f6-5ca882c3a93c" (UID: "dd4bd540-e78e-4939-b8f6-5ca882c3a93c"). InnerVolumeSpecName "kube-api-access-sr4qn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:23:06.082662 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:06.082574    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr4qn\" (UniqueName: \"kubernetes.io/projected/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-kube-api-access-sr4qn\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:23:06.082662 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:06.082608    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:23:06.082662 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:06.082620    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd4bd540-e78e-4939-b8f6-5ca882c3a93c-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:23:06.280581 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:06.280546    2582 generic.go:356] "Generic (PLEG): container finished" podID="dd4bd540-e78e-4939-b8f6-5ca882c3a93c" containerID="2e3d99e826af9a34d3b5f804dddde6bbfdf091fa6cb1bb7f5bf01ec560a57e9b" exitCode=0
Sep 11 13:23:06.280581 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:06.280584    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q" event={"ID":"dd4bd540-e78e-4939-b8f6-5ca882c3a93c","Type":"ContainerDied","Data":"2e3d99e826af9a34d3b5f804dddde6bbfdf091fa6cb1bb7f5bf01ec560a57e9b"}
Sep 11 13:23:06.280802 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:06.280593    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"
Sep 11 13:23:06.280802 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:06.280628    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q" event={"ID":"dd4bd540-e78e-4939-b8f6-5ca882c3a93c","Type":"ContainerDied","Data":"7fb0250967095a1c8d810a124782778ed38321b8415359a3460a608932061dc3"}
Sep 11 13:23:06.280802 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:06.280651    2582 scope.go:117] "RemoveContainer" containerID="2e3d99e826af9a34d3b5f804dddde6bbfdf091fa6cb1bb7f5bf01ec560a57e9b"
Sep 11 13:23:06.290313 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:06.290295    2582 scope.go:117] "RemoveContainer" containerID="2e3d99e826af9a34d3b5f804dddde6bbfdf091fa6cb1bb7f5bf01ec560a57e9b"
Sep 11 13:23:06.290585 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:23:06.290565    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e3d99e826af9a34d3b5f804dddde6bbfdf091fa6cb1bb7f5bf01ec560a57e9b\": container with ID starting with 2e3d99e826af9a34d3b5f804dddde6bbfdf091fa6cb1bb7f5bf01ec560a57e9b not found: ID does not exist" containerID="2e3d99e826af9a34d3b5f804dddde6bbfdf091fa6cb1bb7f5bf01ec560a57e9b"
Sep 11 13:23:06.290633 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:06.290595    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e3d99e826af9a34d3b5f804dddde6bbfdf091fa6cb1bb7f5bf01ec560a57e9b"} err="failed to get container status \"2e3d99e826af9a34d3b5f804dddde6bbfdf091fa6cb1bb7f5bf01ec560a57e9b\": rpc error: code = NotFound desc = could not find container \"2e3d99e826af9a34d3b5f804dddde6bbfdf091fa6cb1bb7f5bf01ec560a57e9b\": container with ID starting with 2e3d99e826af9a34d3b5f804dddde6bbfdf091fa6cb1bb7f5bf01ec560a57e9b not found: ID does not exist"
Sep 11 13:23:06.305485 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:06.305460    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"]
Sep 11 13:23:06.313902 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:06.313878    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cc857f-c49678896-rd78q"]
Sep 11 13:23:06.476155 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:23:06.476118    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd4bd540-e78e-4939-b8f6-5ca882c3a93c" path="/var/lib/kubelet/pods/dd4bd540-e78e-4939-b8f6-5ca882c3a93c/volumes"
Sep 11 13:24:12.683716 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:12.683633    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"]
Sep 11 13:24:12.684658 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:12.683842    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g" podUID="4ecd8b7f-48b1-40f4-b39e-687b5c0958c8" containerName="maas-api" containerID="cri-o://90c6317460364a9c3f130effc2e8a8664bebb931480606b79098f2b385cc2bdd" gracePeriod=30
Sep 11 13:24:12.943102 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:12.943038    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:24:13.046755 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.046715    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-openshift-service-ca\") pod \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") "
Sep 11 13:24:13.046935 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.046784    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x92jh\" (UniqueName: \"kubernetes.io/projected/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-kube-api-access-x92jh\") pod \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") "
Sep 11 13:24:13.046935 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.046813    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-maas-api-tls\") pod \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") "
Sep 11 13:24:13.046935 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.046842    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-metrics-certs\") pod \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\" (UID: \"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8\") "
Sep 11 13:24:13.047644 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.047446    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4ecd8b7f-48b1-40f4-b39e-687b5c0958c8" (UID: "4ecd8b7f-48b1-40f4-b39e-687b5c0958c8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:13.047799 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.047751    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:24:13.056496 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.050193    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-kube-api-access-x92jh" (OuterVolumeSpecName: "kube-api-access-x92jh") pod "4ecd8b7f-48b1-40f4-b39e-687b5c0958c8" (UID: "4ecd8b7f-48b1-40f4-b39e-687b5c0958c8"). InnerVolumeSpecName "kube-api-access-x92jh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:13.060142 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.060099    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4ecd8b7f-48b1-40f4-b39e-687b5c0958c8" (UID: "4ecd8b7f-48b1-40f4-b39e-687b5c0958c8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:13.061009 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.060974    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4ecd8b7f-48b1-40f4-b39e-687b5c0958c8" (UID: "4ecd8b7f-48b1-40f4-b39e-687b5c0958c8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:13.148499 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.148464    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x92jh\" (UniqueName: \"kubernetes.io/projected/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-kube-api-access-x92jh\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:24:13.148499 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.148494    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:24:13.148499 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.148503    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:24:13.646170 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.646135    2582 generic.go:356] "Generic (PLEG): container finished" podID="4ecd8b7f-48b1-40f4-b39e-687b5c0958c8" containerID="90c6317460364a9c3f130effc2e8a8664bebb931480606b79098f2b385cc2bdd" exitCode=0
Sep 11 13:24:13.646367 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.646195    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"
Sep 11 13:24:13.646367 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.646221    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g" event={"ID":"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8","Type":"ContainerDied","Data":"90c6317460364a9c3f130effc2e8a8664bebb931480606b79098f2b385cc2bdd"}
Sep 11 13:24:13.646367 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.646276    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g" event={"ID":"4ecd8b7f-48b1-40f4-b39e-687b5c0958c8","Type":"ContainerDied","Data":"8ec850f78001acd4b844fae3cbab6dbf87e0cc6f9215485184c3ab70ab83f070"}
Sep 11 13:24:13.646367 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.646299    2582 scope.go:117] "RemoveContainer" containerID="90c6317460364a9c3f130effc2e8a8664bebb931480606b79098f2b385cc2bdd"
Sep 11 13:24:13.657487 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.657464    2582 scope.go:117] "RemoveContainer" containerID="90c6317460364a9c3f130effc2e8a8664bebb931480606b79098f2b385cc2bdd"
Sep 11 13:24:13.657811 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:24:13.657789    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90c6317460364a9c3f130effc2e8a8664bebb931480606b79098f2b385cc2bdd\": container with ID starting with 90c6317460364a9c3f130effc2e8a8664bebb931480606b79098f2b385cc2bdd not found: ID does not exist" containerID="90c6317460364a9c3f130effc2e8a8664bebb931480606b79098f2b385cc2bdd"
Sep 11 13:24:13.657895 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.657824    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90c6317460364a9c3f130effc2e8a8664bebb931480606b79098f2b385cc2bdd"} err="failed to get container status \"90c6317460364a9c3f130effc2e8a8664bebb931480606b79098f2b385cc2bdd\": rpc error: code = NotFound desc = could not find container \"90c6317460364a9c3f130effc2e8a8664bebb931480606b79098f2b385cc2bdd\": container with ID starting with 90c6317460364a9c3f130effc2e8a8664bebb931480606b79098f2b385cc2bdd not found: ID does not exist"
Sep 11 13:24:13.670532 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.670507    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"]
Sep 11 13:24:13.675332 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:13.675310    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e3f42f-56ffc46bf5-88x2g"]
Sep 11 13:24:14.477082 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:14.477046    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ecd8b7f-48b1-40f4-b39e-687b5c0958c8" path="/var/lib/kubelet/pods/4ecd8b7f-48b1-40f4-b39e-687b5c0958c8/volumes"
Sep 11 13:24:30.714443 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:30.711905    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"]
Sep 11 13:24:30.714443 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:30.712184    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j" podUID="29e1a8f2-e0b4-4679-acf4-1dadecfd0133" containerName="maas-api" containerID="cri-o://f3be677a00e3d05dfb1255682ce9ed992ac9170161839fa56783f217ae78cc3b" gracePeriod=30
Sep 11 13:24:30.974571 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:30.974541    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:24:31.121530 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.121488    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kzx8\" (UniqueName: \"kubernetes.io/projected/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-kube-api-access-6kzx8\") pod \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") "
Sep 11 13:24:31.121737 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.121562    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-metrics-certs\") pod \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") "
Sep 11 13:24:31.121737 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.121595    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-maas-api-tls\") pod \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") "
Sep 11 13:24:31.121737 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.121620    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-openshift-service-ca\") pod \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\" (UID: \"29e1a8f2-e0b4-4679-acf4-1dadecfd0133\") "
Sep 11 13:24:31.122088 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.122053    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "29e1a8f2-e0b4-4679-acf4-1dadecfd0133" (UID: "29e1a8f2-e0b4-4679-acf4-1dadecfd0133"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:31.123894 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.123859    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-kube-api-access-6kzx8" (OuterVolumeSpecName: "kube-api-access-6kzx8") pod "29e1a8f2-e0b4-4679-acf4-1dadecfd0133" (UID: "29e1a8f2-e0b4-4679-acf4-1dadecfd0133"). InnerVolumeSpecName "kube-api-access-6kzx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:31.123995 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.123903    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "29e1a8f2-e0b4-4679-acf4-1dadecfd0133" (UID: "29e1a8f2-e0b4-4679-acf4-1dadecfd0133"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:31.123995 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.123924    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "29e1a8f2-e0b4-4679-acf4-1dadecfd0133" (UID: "29e1a8f2-e0b4-4679-acf4-1dadecfd0133"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:31.222912 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.222872    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:24:31.222912 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.222907    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:24:31.222912 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.222917    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:24:31.222912 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.222925    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kzx8\" (UniqueName: \"kubernetes.io/projected/29e1a8f2-e0b4-4679-acf4-1dadecfd0133-kube-api-access-6kzx8\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:24:31.746390 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.746352    2582 generic.go:356] "Generic (PLEG): container finished" podID="29e1a8f2-e0b4-4679-acf4-1dadecfd0133" containerID="f3be677a00e3d05dfb1255682ce9ed992ac9170161839fa56783f217ae78cc3b" exitCode=0
Sep 11 13:24:31.746390 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.746394    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j" event={"ID":"29e1a8f2-e0b4-4679-acf4-1dadecfd0133","Type":"ContainerDied","Data":"f3be677a00e3d05dfb1255682ce9ed992ac9170161839fa56783f217ae78cc3b"}
Sep 11 13:24:31.746911 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.746402    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"
Sep 11 13:24:31.746911 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.746454    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j" event={"ID":"29e1a8f2-e0b4-4679-acf4-1dadecfd0133","Type":"ContainerDied","Data":"2e9a441f5d2cdc20d4b151501b1cc666072c1870a9dd5d9e59a385347992eded"}
Sep 11 13:24:31.746911 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.746478    2582 scope.go:117] "RemoveContainer" containerID="f3be677a00e3d05dfb1255682ce9ed992ac9170161839fa56783f217ae78cc3b"
Sep 11 13:24:31.757646 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.757626    2582 scope.go:117] "RemoveContainer" containerID="f3be677a00e3d05dfb1255682ce9ed992ac9170161839fa56783f217ae78cc3b"
Sep 11 13:24:31.757953 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:24:31.757931    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3be677a00e3d05dfb1255682ce9ed992ac9170161839fa56783f217ae78cc3b\": container with ID starting with f3be677a00e3d05dfb1255682ce9ed992ac9170161839fa56783f217ae78cc3b not found: ID does not exist" containerID="f3be677a00e3d05dfb1255682ce9ed992ac9170161839fa56783f217ae78cc3b"
Sep 11 13:24:31.757999 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.757962    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3be677a00e3d05dfb1255682ce9ed992ac9170161839fa56783f217ae78cc3b"} err="failed to get container status \"f3be677a00e3d05dfb1255682ce9ed992ac9170161839fa56783f217ae78cc3b\": rpc error: code = NotFound desc = could not find container \"f3be677a00e3d05dfb1255682ce9ed992ac9170161839fa56783f217ae78cc3b\": container with ID starting with f3be677a00e3d05dfb1255682ce9ed992ac9170161839fa56783f217ae78cc3b not found: ID does not exist"
Sep 11 13:24:31.783429 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.783386    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"]
Sep 11 13:24:31.793982 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:31.793953    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1841da-b8c65578b-cpx2j"]
Sep 11 13:24:32.475997 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:32.475957    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29e1a8f2-e0b4-4679-acf4-1dadecfd0133" path="/var/lib/kubelet/pods/29e1a8f2-e0b4-4679-acf4-1dadecfd0133/volumes"
Sep 11 13:24:49.450039 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.450005    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"]
Sep 11 13:24:49.450513 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.450206    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk" podUID="afb691b2-b7ec-4a41-89b4-850561e49836" containerName="maas-api" containerID="cri-o://d99b0653a1f800f7fb469f8a0a2f8a6b3c0468db60e538744e13f4077b63b5e9" gracePeriod=30
Sep 11 13:24:49.712921 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.712851    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:24:49.792626 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.792585    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afb691b2-b7ec-4a41-89b4-850561e49836-metrics-certs\") pod \"afb691b2-b7ec-4a41-89b4-850561e49836\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") "
Sep 11 13:24:49.792626 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.792619    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afb691b2-b7ec-4a41-89b4-850561e49836-maas-api-tls\") pod \"afb691b2-b7ec-4a41-89b4-850561e49836\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") "
Sep 11 13:24:49.792879 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.792639    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9ndf\" (UniqueName: \"kubernetes.io/projected/afb691b2-b7ec-4a41-89b4-850561e49836-kube-api-access-q9ndf\") pod \"afb691b2-b7ec-4a41-89b4-850561e49836\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") "
Sep 11 13:24:49.792879 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.792664    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afb691b2-b7ec-4a41-89b4-850561e49836-openshift-service-ca\") pod \"afb691b2-b7ec-4a41-89b4-850561e49836\" (UID: \"afb691b2-b7ec-4a41-89b4-850561e49836\") "
Sep 11 13:24:49.793112 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.793084    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afb691b2-b7ec-4a41-89b4-850561e49836-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "afb691b2-b7ec-4a41-89b4-850561e49836" (UID: "afb691b2-b7ec-4a41-89b4-850561e49836"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:49.794838 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.794809    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afb691b2-b7ec-4a41-89b4-850561e49836-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "afb691b2-b7ec-4a41-89b4-850561e49836" (UID: "afb691b2-b7ec-4a41-89b4-850561e49836"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:49.794838 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.794823    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afb691b2-b7ec-4a41-89b4-850561e49836-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "afb691b2-b7ec-4a41-89b4-850561e49836" (UID: "afb691b2-b7ec-4a41-89b4-850561e49836"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:49.794974 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.794865    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afb691b2-b7ec-4a41-89b4-850561e49836-kube-api-access-q9ndf" (OuterVolumeSpecName: "kube-api-access-q9ndf") pod "afb691b2-b7ec-4a41-89b4-850561e49836" (UID: "afb691b2-b7ec-4a41-89b4-850561e49836"). InnerVolumeSpecName "kube-api-access-q9ndf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:49.843768 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.843733    2582 generic.go:356] "Generic (PLEG): container finished" podID="afb691b2-b7ec-4a41-89b4-850561e49836" containerID="d99b0653a1f800f7fb469f8a0a2f8a6b3c0468db60e538744e13f4077b63b5e9" exitCode=0
Sep 11 13:24:49.843925 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.843777    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"
Sep 11 13:24:49.843925 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.843801    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk" event={"ID":"afb691b2-b7ec-4a41-89b4-850561e49836","Type":"ContainerDied","Data":"d99b0653a1f800f7fb469f8a0a2f8a6b3c0468db60e538744e13f4077b63b5e9"}
Sep 11 13:24:49.843925 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.843845    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk" event={"ID":"afb691b2-b7ec-4a41-89b4-850561e49836","Type":"ContainerDied","Data":"3050111286b1e08a2b92ef8201423ba592f8a43b3193d452711f32269198688d"}
Sep 11 13:24:49.843925 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.843866    2582 scope.go:117] "RemoveContainer" containerID="d99b0653a1f800f7fb469f8a0a2f8a6b3c0468db60e538744e13f4077b63b5e9"
Sep 11 13:24:49.853694 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.853672    2582 scope.go:117] "RemoveContainer" containerID="d99b0653a1f800f7fb469f8a0a2f8a6b3c0468db60e538744e13f4077b63b5e9"
Sep 11 13:24:49.854030 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:24:49.854003    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d99b0653a1f800f7fb469f8a0a2f8a6b3c0468db60e538744e13f4077b63b5e9\": container with ID starting with d99b0653a1f800f7fb469f8a0a2f8a6b3c0468db60e538744e13f4077b63b5e9 not found: ID does not exist" containerID="d99b0653a1f800f7fb469f8a0a2f8a6b3c0468db60e538744e13f4077b63b5e9"
Sep 11 13:24:49.854127 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.854042    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d99b0653a1f800f7fb469f8a0a2f8a6b3c0468db60e538744e13f4077b63b5e9"} err="failed to get container status \"d99b0653a1f800f7fb469f8a0a2f8a6b3c0468db60e538744e13f4077b63b5e9\": rpc error: code = NotFound desc = could not find container \"d99b0653a1f800f7fb469f8a0a2f8a6b3c0468db60e538744e13f4077b63b5e9\": container with ID starting with d99b0653a1f800f7fb469f8a0a2f8a6b3c0468db60e538744e13f4077b63b5e9 not found: ID does not exist"
Sep 11 13:24:49.872643 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.872616    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"]
Sep 11 13:24:49.878945 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.878920    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-82825f-cbc9fb547-x9zxk"]
Sep 11 13:24:49.893626 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.893605    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afb691b2-b7ec-4a41-89b4-850561e49836-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:24:49.893626 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.893625    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afb691b2-b7ec-4a41-89b4-850561e49836-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:24:49.893767 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.893634    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9ndf\" (UniqueName: \"kubernetes.io/projected/afb691b2-b7ec-4a41-89b4-850561e49836-kube-api-access-q9ndf\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:24:49.893767 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:49.893644    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afb691b2-b7ec-4a41-89b4-850561e49836-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:24:50.477071 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:50.477024    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afb691b2-b7ec-4a41-89b4-850561e49836" path="/var/lib/kubelet/pods/afb691b2-b7ec-4a41-89b4-850561e49836/volumes"
Sep 11 13:24:59.878551 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:59.878510    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65b89bb8c4-d7jx8"]
Sep 11 13:24:59.879047 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:24:59.878743    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65b89bb8c4-d7jx8" podUID="8c2abef0-5a85-40d5-8fec-9db765343ceb" containerName="manager" containerID="cri-o://e069d519d91cccf027660b14a5d3788f0f4558dd506f26b21cc33bae441e2f0d" gracePeriod=30
Sep 11 13:25:00.135184 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.135116    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65b89bb8c4-d7jx8"
Sep 11 13:25:00.195804 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.195773    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c2abef0-5a85-40d5-8fec-9db765343ceb-metrics-certs\") pod \"8c2abef0-5a85-40d5-8fec-9db765343ceb\" (UID: \"8c2abef0-5a85-40d5-8fec-9db765343ceb\") "
Sep 11 13:25:00.195962 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.195857    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c2abef0-5a85-40d5-8fec-9db765343ceb-webhook-cert\") pod \"8c2abef0-5a85-40d5-8fec-9db765343ceb\" (UID: \"8c2abef0-5a85-40d5-8fec-9db765343ceb\") "
Sep 11 13:25:00.195962 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.195917    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgwqb\" (UniqueName: \"kubernetes.io/projected/8c2abef0-5a85-40d5-8fec-9db765343ceb-kube-api-access-sgwqb\") pod \"8c2abef0-5a85-40d5-8fec-9db765343ceb\" (UID: \"8c2abef0-5a85-40d5-8fec-9db765343ceb\") "
Sep 11 13:25:00.198053 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.198015    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c2abef0-5a85-40d5-8fec-9db765343ceb-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8c2abef0-5a85-40d5-8fec-9db765343ceb" (UID: "8c2abef0-5a85-40d5-8fec-9db765343ceb"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:25:00.198206 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.198045    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c2abef0-5a85-40d5-8fec-9db765343ceb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8c2abef0-5a85-40d5-8fec-9db765343ceb" (UID: "8c2abef0-5a85-40d5-8fec-9db765343ceb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:25:00.198206 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.198096    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c2abef0-5a85-40d5-8fec-9db765343ceb-kube-api-access-sgwqb" (OuterVolumeSpecName: "kube-api-access-sgwqb") pod "8c2abef0-5a85-40d5-8fec-9db765343ceb" (UID: "8c2abef0-5a85-40d5-8fec-9db765343ceb"). InnerVolumeSpecName "kube-api-access-sgwqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:25:00.297411 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.297368    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgwqb\" (UniqueName: \"kubernetes.io/projected/8c2abef0-5a85-40d5-8fec-9db765343ceb-kube-api-access-sgwqb\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:25:00.297411 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.297406    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c2abef0-5a85-40d5-8fec-9db765343ceb-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:25:00.297411 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.297416    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c2abef0-5a85-40d5-8fec-9db765343ceb-webhook-cert\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:25:00.906780 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.906682    2582 generic.go:356] "Generic (PLEG): container finished" podID="8c2abef0-5a85-40d5-8fec-9db765343ceb" containerID="e069d519d91cccf027660b14a5d3788f0f4558dd506f26b21cc33bae441e2f0d" exitCode=0
Sep 11 13:25:00.906780 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.906734    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65b89bb8c4-d7jx8"
Sep 11 13:25:00.906780 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.906773    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65b89bb8c4-d7jx8" event={"ID":"8c2abef0-5a85-40d5-8fec-9db765343ceb","Type":"ContainerDied","Data":"e069d519d91cccf027660b14a5d3788f0f4558dd506f26b21cc33bae441e2f0d"}
Sep 11 13:25:00.907329 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.906825    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65b89bb8c4-d7jx8" event={"ID":"8c2abef0-5a85-40d5-8fec-9db765343ceb","Type":"ContainerDied","Data":"5533e3b3d5389c21e340d124e06ab8554a521f14ac38b45e66f11889b850da49"}
Sep 11 13:25:00.907329 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.906849    2582 scope.go:117] "RemoveContainer" containerID="e069d519d91cccf027660b14a5d3788f0f4558dd506f26b21cc33bae441e2f0d"
Sep 11 13:25:00.916141 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.916118    2582 scope.go:117] "RemoveContainer" containerID="e069d519d91cccf027660b14a5d3788f0f4558dd506f26b21cc33bae441e2f0d"
Sep 11 13:25:00.916497 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:25:00.916474    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e069d519d91cccf027660b14a5d3788f0f4558dd506f26b21cc33bae441e2f0d\": container with ID starting with e069d519d91cccf027660b14a5d3788f0f4558dd506f26b21cc33bae441e2f0d not found: ID does not exist" containerID="e069d519d91cccf027660b14a5d3788f0f4558dd506f26b21cc33bae441e2f0d"
Sep 11 13:25:00.916551 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.916506    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e069d519d91cccf027660b14a5d3788f0f4558dd506f26b21cc33bae441e2f0d"} err="failed to get container status \"e069d519d91cccf027660b14a5d3788f0f4558dd506f26b21cc33bae441e2f0d\": rpc error: code = NotFound desc = could not find container \"e069d519d91cccf027660b14a5d3788f0f4558dd506f26b21cc33bae441e2f0d\": container with ID starting with e069d519d91cccf027660b14a5d3788f0f4558dd506f26b21cc33bae441e2f0d not found: ID does not exist"
Sep 11 13:25:00.927008 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.926978    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65b89bb8c4-d7jx8"]
Sep 11 13:25:00.931119 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:00.931091    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65b89bb8c4-d7jx8"]
Sep 11 13:25:02.476531 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:25:02.476499    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c2abef0-5a85-40d5-8fec-9db765343ceb" path="/var/lib/kubelet/pods/8c2abef0-5a85-40d5-8fec-9db765343ceb/volumes"
Sep 11 13:26:01.779397 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.779358    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65b89bb8c4-2qsgk"]
Sep 11 13:26:01.779966 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.779948    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd4bd540-e78e-4939-b8f6-5ca882c3a93c" containerName="maas-api"
Sep 11 13:26:01.779966 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.779967    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd4bd540-e78e-4939-b8f6-5ca882c3a93c" containerName="maas-api"
Sep 11 13:26:01.780071 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.779978    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29e1a8f2-e0b4-4679-acf4-1dadecfd0133" containerName="maas-api"
Sep 11 13:26:01.780071 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.779983    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="29e1a8f2-e0b4-4679-acf4-1dadecfd0133" containerName="maas-api"
Sep 11 13:26:01.780071 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.779998    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ecd8b7f-48b1-40f4-b39e-687b5c0958c8" containerName="maas-api"
Sep 11 13:26:01.780071 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.780004    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ecd8b7f-48b1-40f4-b39e-687b5c0958c8" containerName="maas-api"
Sep 11 13:26:01.780071 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.780010    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e45ab133-d279-41a5-8ee6-2ee43c987a58" containerName="maas-api"
Sep 11 13:26:01.780071 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.780015    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e45ab133-d279-41a5-8ee6-2ee43c987a58" containerName="maas-api"
Sep 11 13:26:01.780071 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.780026    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afb691b2-b7ec-4a41-89b4-850561e49836" containerName="maas-api"
Sep 11 13:26:01.780071 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.780032    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="afb691b2-b7ec-4a41-89b4-850561e49836" containerName="maas-api"
Sep 11 13:26:01.780071 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.780041    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c2abef0-5a85-40d5-8fec-9db765343ceb" containerName="manager"
Sep 11 13:26:01.780071 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.780046    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c2abef0-5a85-40d5-8fec-9db765343ceb" containerName="manager"
Sep 11 13:26:01.780372 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.780102    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ecd8b7f-48b1-40f4-b39e-687b5c0958c8" containerName="maas-api"
Sep 11 13:26:01.780372 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.780110    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="afb691b2-b7ec-4a41-89b4-850561e49836" containerName="maas-api"
Sep 11 13:26:01.780372 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.780117    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e45ab133-d279-41a5-8ee6-2ee43c987a58" containerName="maas-api"
Sep 11 13:26:01.780372 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.780125    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c2abef0-5a85-40d5-8fec-9db765343ceb" containerName="manager"
Sep 11 13:26:01.780372 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.780133    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd4bd540-e78e-4939-b8f6-5ca882c3a93c" containerName="maas-api"
Sep 11 13:26:01.780372 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.780140    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="29e1a8f2-e0b4-4679-acf4-1dadecfd0133" containerName="maas-api"
Sep 11 13:26:01.783129 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.783114    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65b89bb8c4-2qsgk"
Sep 11 13:26:01.787223 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.787191    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dmcj8\""
Sep 11 13:26:01.787223 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.787196    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 13:26:01.787433 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.787227    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 13:26:01.787433 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.787306    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 13:26:01.795739 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.795717    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65b89bb8c4-2qsgk"]
Sep 11 13:26:01.802335 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.802311    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af174ba2-02e2-4e8e-9f02-77bfe010ac42-webhook-cert\") pod \"maas-controller-65b89bb8c4-2qsgk\" (UID: \"af174ba2-02e2-4e8e-9f02-77bfe010ac42\") " pod="opendatahub/maas-controller-65b89bb8c4-2qsgk"
Sep 11 13:26:01.802447 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.802353    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbhk7\" (UniqueName: \"kubernetes.io/projected/af174ba2-02e2-4e8e-9f02-77bfe010ac42-kube-api-access-lbhk7\") pod \"maas-controller-65b89bb8c4-2qsgk\" (UID: \"af174ba2-02e2-4e8e-9f02-77bfe010ac42\") " pod="opendatahub/maas-controller-65b89bb8c4-2qsgk"
Sep 11 13:26:01.802447 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.802388    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af174ba2-02e2-4e8e-9f02-77bfe010ac42-metrics-certs\") pod \"maas-controller-65b89bb8c4-2qsgk\" (UID: \"af174ba2-02e2-4e8e-9f02-77bfe010ac42\") " pod="opendatahub/maas-controller-65b89bb8c4-2qsgk"
Sep 11 13:26:01.903330 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.903281    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af174ba2-02e2-4e8e-9f02-77bfe010ac42-webhook-cert\") pod \"maas-controller-65b89bb8c4-2qsgk\" (UID: \"af174ba2-02e2-4e8e-9f02-77bfe010ac42\") " pod="opendatahub/maas-controller-65b89bb8c4-2qsgk"
Sep 11 13:26:01.903539 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.903346    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbhk7\" (UniqueName: \"kubernetes.io/projected/af174ba2-02e2-4e8e-9f02-77bfe010ac42-kube-api-access-lbhk7\") pod \"maas-controller-65b89bb8c4-2qsgk\" (UID: \"af174ba2-02e2-4e8e-9f02-77bfe010ac42\") " pod="opendatahub/maas-controller-65b89bb8c4-2qsgk"
Sep 11 13:26:01.903539 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.903380    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af174ba2-02e2-4e8e-9f02-77bfe010ac42-metrics-certs\") pod \"maas-controller-65b89bb8c4-2qsgk\" (UID: \"af174ba2-02e2-4e8e-9f02-77bfe010ac42\") " pod="opendatahub/maas-controller-65b89bb8c4-2qsgk"
Sep 11 13:26:01.905790 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.905759    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af174ba2-02e2-4e8e-9f02-77bfe010ac42-metrics-certs\") pod \"maas-controller-65b89bb8c4-2qsgk\" (UID: \"af174ba2-02e2-4e8e-9f02-77bfe010ac42\") " pod="opendatahub/maas-controller-65b89bb8c4-2qsgk"
Sep 11 13:26:01.905934 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.905911    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af174ba2-02e2-4e8e-9f02-77bfe010ac42-webhook-cert\") pod \"maas-controller-65b89bb8c4-2qsgk\" (UID: \"af174ba2-02e2-4e8e-9f02-77bfe010ac42\") " pod="opendatahub/maas-controller-65b89bb8c4-2qsgk"
Sep 11 13:26:01.912464 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:01.912444    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbhk7\" (UniqueName: \"kubernetes.io/projected/af174ba2-02e2-4e8e-9f02-77bfe010ac42-kube-api-access-lbhk7\") pod \"maas-controller-65b89bb8c4-2qsgk\" (UID: \"af174ba2-02e2-4e8e-9f02-77bfe010ac42\") " pod="opendatahub/maas-controller-65b89bb8c4-2qsgk"
Sep 11 13:26:02.094051 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:02.093975    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65b89bb8c4-2qsgk"
Sep 11 13:26:02.365179 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:02.365150    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65b89bb8c4-2qsgk"]
Sep 11 13:26:02.366990 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:26:02.366957    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf174ba2_02e2_4e8e_9f02_77bfe010ac42.slice/crio-a8775743f10a51fea533555a61854c48853f86e6320c2cfa222d5f8fa340a55b WatchSource:0}: Error finding container a8775743f10a51fea533555a61854c48853f86e6320c2cfa222d5f8fa340a55b: Status 404 returned error can't find the container with id a8775743f10a51fea533555a61854c48853f86e6320c2cfa222d5f8fa340a55b
Sep 11 13:26:02.368316 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:02.368286    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:26:03.226136 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:03.226101    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65b89bb8c4-2qsgk" event={"ID":"af174ba2-02e2-4e8e-9f02-77bfe010ac42","Type":"ContainerStarted","Data":"9f917d0ad7437727837cf40b4dca3510f2d6805d4333aa124eee38eaf2e4e2a4"}
Sep 11 13:26:03.226614 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:03.226273    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65b89bb8c4-2qsgk"
Sep 11 13:26:03.226614 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:03.226294    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65b89bb8c4-2qsgk" event={"ID":"af174ba2-02e2-4e8e-9f02-77bfe010ac42","Type":"ContainerStarted","Data":"a8775743f10a51fea533555a61854c48853f86e6320c2cfa222d5f8fa340a55b"}
Sep 11 13:26:03.247920 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:03.247871    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65b89bb8c4-2qsgk" podStartSLOduration=1.808589878 podStartE2EDuration="2.247855401s" totalImagesPullingTime="439.265523ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:03.245952704 +0000 UTC m=+1817.185654038" watchObservedRunningTime="2026-09-11 13:26:03.247855401 +0000 UTC m=+1817.187556734"
Sep 11 13:26:14.233895 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:14.233861    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65b89bb8c4-2qsgk"
Sep 11 13:26:23.729915 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:23.729881    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65b89bb8c4-2qsgk"]
Sep 11 13:26:23.734641 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:23.730085    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65b89bb8c4-2qsgk" podUID="af174ba2-02e2-4e8e-9f02-77bfe010ac42" containerName="manager" containerID="cri-o://9f917d0ad7437727837cf40b4dca3510f2d6805d4333aa124eee38eaf2e4e2a4" gracePeriod=30
Sep 11 13:26:23.978816 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:23.978786    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65b89bb8c4-2qsgk"
Sep 11 13:26:24.112650 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.112561    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af174ba2-02e2-4e8e-9f02-77bfe010ac42-webhook-cert\") pod \"af174ba2-02e2-4e8e-9f02-77bfe010ac42\" (UID: \"af174ba2-02e2-4e8e-9f02-77bfe010ac42\") "
Sep 11 13:26:24.112650 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.112597    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af174ba2-02e2-4e8e-9f02-77bfe010ac42-metrics-certs\") pod \"af174ba2-02e2-4e8e-9f02-77bfe010ac42\" (UID: \"af174ba2-02e2-4e8e-9f02-77bfe010ac42\") "
Sep 11 13:26:24.112885 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.112664    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbhk7\" (UniqueName: \"kubernetes.io/projected/af174ba2-02e2-4e8e-9f02-77bfe010ac42-kube-api-access-lbhk7\") pod \"af174ba2-02e2-4e8e-9f02-77bfe010ac42\" (UID: \"af174ba2-02e2-4e8e-9f02-77bfe010ac42\") "
Sep 11 13:26:24.114633 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.114594    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af174ba2-02e2-4e8e-9f02-77bfe010ac42-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "af174ba2-02e2-4e8e-9f02-77bfe010ac42" (UID: "af174ba2-02e2-4e8e-9f02-77bfe010ac42"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:26:24.114775 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.114663    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af174ba2-02e2-4e8e-9f02-77bfe010ac42-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "af174ba2-02e2-4e8e-9f02-77bfe010ac42" (UID: "af174ba2-02e2-4e8e-9f02-77bfe010ac42"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:26:24.114775 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.114710    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af174ba2-02e2-4e8e-9f02-77bfe010ac42-kube-api-access-lbhk7" (OuterVolumeSpecName: "kube-api-access-lbhk7") pod "af174ba2-02e2-4e8e-9f02-77bfe010ac42" (UID: "af174ba2-02e2-4e8e-9f02-77bfe010ac42"). InnerVolumeSpecName "kube-api-access-lbhk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:26:24.213526 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.213499    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbhk7\" (UniqueName: \"kubernetes.io/projected/af174ba2-02e2-4e8e-9f02-77bfe010ac42-kube-api-access-lbhk7\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:26:24.213526 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.213523    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af174ba2-02e2-4e8e-9f02-77bfe010ac42-webhook-cert\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:26:24.213526 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.213533    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af174ba2-02e2-4e8e-9f02-77bfe010ac42-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:26:24.332948 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.332912    2582 generic.go:356] "Generic (PLEG): container finished" podID="af174ba2-02e2-4e8e-9f02-77bfe010ac42" containerID="9f917d0ad7437727837cf40b4dca3510f2d6805d4333aa124eee38eaf2e4e2a4" exitCode=0
Sep 11 13:26:24.333098 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.332963    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65b89bb8c4-2qsgk"
Sep 11 13:26:24.333098 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.332965    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65b89bb8c4-2qsgk" event={"ID":"af174ba2-02e2-4e8e-9f02-77bfe010ac42","Type":"ContainerDied","Data":"9f917d0ad7437727837cf40b4dca3510f2d6805d4333aa124eee38eaf2e4e2a4"}
Sep 11 13:26:24.333098 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.333090    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65b89bb8c4-2qsgk" event={"ID":"af174ba2-02e2-4e8e-9f02-77bfe010ac42","Type":"ContainerDied","Data":"a8775743f10a51fea533555a61854c48853f86e6320c2cfa222d5f8fa340a55b"}
Sep 11 13:26:24.333213 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.333117    2582 scope.go:117] "RemoveContainer" containerID="9f917d0ad7437727837cf40b4dca3510f2d6805d4333aa124eee38eaf2e4e2a4"
Sep 11 13:26:24.344942 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.344924    2582 scope.go:117] "RemoveContainer" containerID="9f917d0ad7437727837cf40b4dca3510f2d6805d4333aa124eee38eaf2e4e2a4"
Sep 11 13:26:24.345199 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:26:24.345174    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f917d0ad7437727837cf40b4dca3510f2d6805d4333aa124eee38eaf2e4e2a4\": container with ID starting with 9f917d0ad7437727837cf40b4dca3510f2d6805d4333aa124eee38eaf2e4e2a4 not found: ID does not exist" containerID="9f917d0ad7437727837cf40b4dca3510f2d6805d4333aa124eee38eaf2e4e2a4"
Sep 11 13:26:24.345246 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.345211    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f917d0ad7437727837cf40b4dca3510f2d6805d4333aa124eee38eaf2e4e2a4"} err="failed to get container status \"9f917d0ad7437727837cf40b4dca3510f2d6805d4333aa124eee38eaf2e4e2a4\": rpc error: code = NotFound desc = could not find container \"9f917d0ad7437727837cf40b4dca3510f2d6805d4333aa124eee38eaf2e4e2a4\": container with ID starting with 9f917d0ad7437727837cf40b4dca3510f2d6805d4333aa124eee38eaf2e4e2a4 not found: ID does not exist"
Sep 11 13:26:24.363178 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.363119    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65b89bb8c4-2qsgk"]
Sep 11 13:26:24.365678 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.365652    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65b89bb8c4-2qsgk"]
Sep 11 13:26:24.476107 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:24.476076    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af174ba2-02e2-4e8e-9f02-77bfe010ac42" path="/var/lib/kubelet/pods/af174ba2-02e2-4e8e-9f02-77bfe010ac42/volumes"
Sep 11 13:26:25.296058 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.296025    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65b89bb8c4-ktsgf"]
Sep 11 13:26:25.296526 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.296511    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af174ba2-02e2-4e8e-9f02-77bfe010ac42" containerName="manager"
Sep 11 13:26:25.296526 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.296528    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="af174ba2-02e2-4e8e-9f02-77bfe010ac42" containerName="manager"
Sep 11 13:26:25.296618 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.296592    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="af174ba2-02e2-4e8e-9f02-77bfe010ac42" containerName="manager"
Sep 11 13:26:25.300958 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.300939    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65b89bb8c4-ktsgf"
Sep 11 13:26:25.304182 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.304163    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 13:26:25.304840 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.304819    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 13:26:25.305468 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.305445    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 13:26:25.305580 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.305513    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dmcj8\""
Sep 11 13:26:25.310025 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.310004    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65b89bb8c4-ktsgf"]
Sep 11 13:26:25.425834 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.425804    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8e53bd3-2211-4c00-8fea-5726fe325da8-webhook-cert\") pod \"maas-controller-65b89bb8c4-ktsgf\" (UID: \"d8e53bd3-2211-4c00-8fea-5726fe325da8\") " pod="opendatahub/maas-controller-65b89bb8c4-ktsgf"
Sep 11 13:26:25.425996 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.425839    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8e53bd3-2211-4c00-8fea-5726fe325da8-metrics-certs\") pod \"maas-controller-65b89bb8c4-ktsgf\" (UID: \"d8e53bd3-2211-4c00-8fea-5726fe325da8\") " pod="opendatahub/maas-controller-65b89bb8c4-ktsgf"
Sep 11 13:26:25.425996 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.425886    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkgpl\" (UniqueName: \"kubernetes.io/projected/d8e53bd3-2211-4c00-8fea-5726fe325da8-kube-api-access-rkgpl\") pod \"maas-controller-65b89bb8c4-ktsgf\" (UID: \"d8e53bd3-2211-4c00-8fea-5726fe325da8\") " pod="opendatahub/maas-controller-65b89bb8c4-ktsgf"
Sep 11 13:26:25.526472 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.526415    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkgpl\" (UniqueName: \"kubernetes.io/projected/d8e53bd3-2211-4c00-8fea-5726fe325da8-kube-api-access-rkgpl\") pod \"maas-controller-65b89bb8c4-ktsgf\" (UID: \"d8e53bd3-2211-4c00-8fea-5726fe325da8\") " pod="opendatahub/maas-controller-65b89bb8c4-ktsgf"
Sep 11 13:26:25.526653 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.526560    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8e53bd3-2211-4c00-8fea-5726fe325da8-webhook-cert\") pod \"maas-controller-65b89bb8c4-ktsgf\" (UID: \"d8e53bd3-2211-4c00-8fea-5726fe325da8\") " pod="opendatahub/maas-controller-65b89bb8c4-ktsgf"
Sep 11 13:26:25.526653 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.526601    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8e53bd3-2211-4c00-8fea-5726fe325da8-metrics-certs\") pod \"maas-controller-65b89bb8c4-ktsgf\" (UID: \"d8e53bd3-2211-4c00-8fea-5726fe325da8\") " pod="opendatahub/maas-controller-65b89bb8c4-ktsgf"
Sep 11 13:26:25.529016 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.528987    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8e53bd3-2211-4c00-8fea-5726fe325da8-webhook-cert\") pod \"maas-controller-65b89bb8c4-ktsgf\" (UID: \"d8e53bd3-2211-4c00-8fea-5726fe325da8\") " pod="opendatahub/maas-controller-65b89bb8c4-ktsgf"
Sep 11 13:26:25.529125 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.529053    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8e53bd3-2211-4c00-8fea-5726fe325da8-metrics-certs\") pod \"maas-controller-65b89bb8c4-ktsgf\" (UID: \"d8e53bd3-2211-4c00-8fea-5726fe325da8\") " pod="opendatahub/maas-controller-65b89bb8c4-ktsgf"
Sep 11 13:26:25.535911 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.535891    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkgpl\" (UniqueName: \"kubernetes.io/projected/d8e53bd3-2211-4c00-8fea-5726fe325da8-kube-api-access-rkgpl\") pod \"maas-controller-65b89bb8c4-ktsgf\" (UID: \"d8e53bd3-2211-4c00-8fea-5726fe325da8\") " pod="opendatahub/maas-controller-65b89bb8c4-ktsgf"
Sep 11 13:26:25.613460 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.613366    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65b89bb8c4-ktsgf"
Sep 11 13:26:25.911259 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:25.911178    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65b89bb8c4-ktsgf"]
Sep 11 13:26:25.915150 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:26:25.915126    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8e53bd3_2211_4c00_8fea_5726fe325da8.slice/crio-7b2ff65da4331991c4a55252c4734c66f69e8ee1ce7a05eb61a507bff08f7e30 WatchSource:0}: Error finding container 7b2ff65da4331991c4a55252c4734c66f69e8ee1ce7a05eb61a507bff08f7e30: Status 404 returned error can't find the container with id 7b2ff65da4331991c4a55252c4734c66f69e8ee1ce7a05eb61a507bff08f7e30
Sep 11 13:26:26.347620 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:26.347580    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65b89bb8c4-ktsgf" event={"ID":"d8e53bd3-2211-4c00-8fea-5726fe325da8","Type":"ContainerStarted","Data":"05a34062cd5c2f08eda52376aaeb5bb6f9c1e1a21f6c9e4e468af03e223fb19d"}
Sep 11 13:26:26.348040 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:26.347641    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65b89bb8c4-ktsgf" event={"ID":"d8e53bd3-2211-4c00-8fea-5726fe325da8","Type":"ContainerStarted","Data":"7b2ff65da4331991c4a55252c4734c66f69e8ee1ce7a05eb61a507bff08f7e30"}
Sep 11 13:26:26.348040 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:26.347677    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65b89bb8c4-ktsgf"
Sep 11 13:26:26.384653 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:26.384590    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65b89bb8c4-ktsgf" podStartSLOduration=1.059767669 podStartE2EDuration="1.384572131s" totalImagesPullingTime="324.804462ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:26.382136683 +0000 UTC m=+1840.321838017" watchObservedRunningTime="2026-09-11 13:26:26.384572131 +0000 UTC m=+1840.324273498"
Sep 11 13:26:37.355518 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:26:37.355482    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65b89bb8c4-ktsgf"
Sep 11 13:27:42.783851 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:42.783821    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"]
Sep 11 13:27:42.784359 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:42.783999    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7" podUID="3163422b-8384-4b3f-97bd-830f0ce1af77" containerName="manager" containerID="cri-o://c92fe3f6818047819c510d2417f28feeb28de6cb0dcc53ef5d09605bd09f7f73" gracePeriod=10
Sep 11 13:27:43.044668 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.044606    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"
Sep 11 13:27:43.180169 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.180138    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3163422b-8384-4b3f-97bd-830f0ce1af77-extensions-socket-volume\") pod \"3163422b-8384-4b3f-97bd-830f0ce1af77\" (UID: \"3163422b-8384-4b3f-97bd-830f0ce1af77\") "
Sep 11 13:27:43.180334 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.180260    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4d986\" (UniqueName: \"kubernetes.io/projected/3163422b-8384-4b3f-97bd-830f0ce1af77-kube-api-access-4d986\") pod \"3163422b-8384-4b3f-97bd-830f0ce1af77\" (UID: \"3163422b-8384-4b3f-97bd-830f0ce1af77\") "
Sep 11 13:27:43.180594 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.180572    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3163422b-8384-4b3f-97bd-830f0ce1af77-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3163422b-8384-4b3f-97bd-830f0ce1af77" (UID: "3163422b-8384-4b3f-97bd-830f0ce1af77"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:27:43.182132 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.182110    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3163422b-8384-4b3f-97bd-830f0ce1af77-kube-api-access-4d986" (OuterVolumeSpecName: "kube-api-access-4d986") pod "3163422b-8384-4b3f-97bd-830f0ce1af77" (UID: "3163422b-8384-4b3f-97bd-830f0ce1af77"). InnerVolumeSpecName "kube-api-access-4d986". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:27:43.282017 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.281970    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3163422b-8384-4b3f-97bd-830f0ce1af77-extensions-socket-volume\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:27:43.282017 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.282009    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d986\" (UniqueName: \"kubernetes.io/projected/3163422b-8384-4b3f-97bd-830f0ce1af77-kube-api-access-4d986\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:27:43.748935 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.748900    2582 generic.go:356] "Generic (PLEG): container finished" podID="3163422b-8384-4b3f-97bd-830f0ce1af77" containerID="c92fe3f6818047819c510d2417f28feeb28de6cb0dcc53ef5d09605bd09f7f73" exitCode=0
Sep 11 13:27:43.749157 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.748955    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"
Sep 11 13:27:43.749157 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.748988    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7" event={"ID":"3163422b-8384-4b3f-97bd-830f0ce1af77","Type":"ContainerDied","Data":"c92fe3f6818047819c510d2417f28feeb28de6cb0dcc53ef5d09605bd09f7f73"}
Sep 11 13:27:43.749157 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.749048    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7" event={"ID":"3163422b-8384-4b3f-97bd-830f0ce1af77","Type":"ContainerDied","Data":"b08e07da0ab4b887713aa2dcfbf482869ec992c4c5a70c4930c09a1375baab4d"}
Sep 11 13:27:43.749157 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.749077    2582 scope.go:117] "RemoveContainer" containerID="c92fe3f6818047819c510d2417f28feeb28de6cb0dcc53ef5d09605bd09f7f73"
Sep 11 13:27:43.759401 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.759383    2582 scope.go:117] "RemoveContainer" containerID="c92fe3f6818047819c510d2417f28feeb28de6cb0dcc53ef5d09605bd09f7f73"
Sep 11 13:27:43.759702 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:27:43.759684    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c92fe3f6818047819c510d2417f28feeb28de6cb0dcc53ef5d09605bd09f7f73\": container with ID starting with c92fe3f6818047819c510d2417f28feeb28de6cb0dcc53ef5d09605bd09f7f73 not found: ID does not exist" containerID="c92fe3f6818047819c510d2417f28feeb28de6cb0dcc53ef5d09605bd09f7f73"
Sep 11 13:27:43.759773 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.759711    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c92fe3f6818047819c510d2417f28feeb28de6cb0dcc53ef5d09605bd09f7f73"} err="failed to get container status \"c92fe3f6818047819c510d2417f28feeb28de6cb0dcc53ef5d09605bd09f7f73\": rpc error: code = NotFound desc = could not find container \"c92fe3f6818047819c510d2417f28feeb28de6cb0dcc53ef5d09605bd09f7f73\": container with ID starting with c92fe3f6818047819c510d2417f28feeb28de6cb0dcc53ef5d09605bd09f7f73 not found: ID does not exist"
Sep 11 13:27:43.772474 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.772450    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"]
Sep 11 13:27:43.776714 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:43.776690    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfnc7"]
Sep 11 13:27:44.475611 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:27:44.475572    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3163422b-8384-4b3f-97bd-830f0ce1af77" path="/var/lib/kubelet/pods/3163422b-8384-4b3f-97bd-830f0ce1af77/volumes"
Sep 11 13:28:43.590641 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:43.590561    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7"]
Sep 11 13:28:43.591022 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:43.590965    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3163422b-8384-4b3f-97bd-830f0ce1af77" containerName="manager"
Sep 11 13:28:43.591022 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:43.590976    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3163422b-8384-4b3f-97bd-830f0ce1af77" containerName="manager"
Sep 11 13:28:43.591092 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:43.591053    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3163422b-8384-4b3f-97bd-830f0ce1af77" containerName="manager"
Sep 11 13:28:43.594290 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:43.594265    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7"
Sep 11 13:28:43.597216 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:43.597196    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tt5b6\""
Sep 11 13:28:43.611987 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:43.611961    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7"]
Sep 11 13:28:43.713146 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:43.713118    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpjbq\" (UniqueName: \"kubernetes.io/projected/7de30b0f-896e-4b4c-a96c-d5468982c05a-kube-api-access-jpjbq\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zqmc7\" (UID: \"7de30b0f-896e-4b4c-a96c-d5468982c05a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7"
Sep 11 13:28:43.713268 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:43.713152    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7de30b0f-896e-4b4c-a96c-d5468982c05a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zqmc7\" (UID: \"7de30b0f-896e-4b4c-a96c-d5468982c05a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7"
Sep 11 13:28:43.813932 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:43.813904    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpjbq\" (UniqueName: \"kubernetes.io/projected/7de30b0f-896e-4b4c-a96c-d5468982c05a-kube-api-access-jpjbq\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zqmc7\" (UID: \"7de30b0f-896e-4b4c-a96c-d5468982c05a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7"
Sep 11 13:28:43.814046 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:43.813938    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7de30b0f-896e-4b4c-a96c-d5468982c05a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zqmc7\" (UID: \"7de30b0f-896e-4b4c-a96c-d5468982c05a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7"
Sep 11 13:28:43.814350 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:43.814331    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7de30b0f-896e-4b4c-a96c-d5468982c05a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zqmc7\" (UID: \"7de30b0f-896e-4b4c-a96c-d5468982c05a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7"
Sep 11 13:28:43.824366 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:43.824331    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpjbq\" (UniqueName: \"kubernetes.io/projected/7de30b0f-896e-4b4c-a96c-d5468982c05a-kube-api-access-jpjbq\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zqmc7\" (UID: \"7de30b0f-896e-4b4c-a96c-d5468982c05a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7"
Sep 11 13:28:43.904814 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:43.904757    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7"
Sep 11 13:28:44.185810 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:44.185788    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7"]
Sep 11 13:28:44.188133 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:28:44.188100    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7de30b0f_896e_4b4c_a96c_d5468982c05a.slice/crio-c6a405850c576b70689344c99751d47168e86399ea41188d0ba04c38ed302db0 WatchSource:0}: Error finding container c6a405850c576b70689344c99751d47168e86399ea41188d0ba04c38ed302db0: Status 404 returned error can't find the container with id c6a405850c576b70689344c99751d47168e86399ea41188d0ba04c38ed302db0
Sep 11 13:28:45.056975 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:45.056939    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7" event={"ID":"7de30b0f-896e-4b4c-a96c-d5468982c05a","Type":"ContainerStarted","Data":"b4c58f34fe42ee89b5f4cab5bbf6a8cd1c5d376780010df8c85cce8e1e730447"}
Sep 11 13:28:45.057434 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:45.057038    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7"
Sep 11 13:28:45.057434 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:45.057067    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7" event={"ID":"7de30b0f-896e-4b4c-a96c-d5468982c05a","Type":"ContainerStarted","Data":"c6a405850c576b70689344c99751d47168e86399ea41188d0ba04c38ed302db0"}
Sep 11 13:28:45.081307 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:45.081259    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7" podStartSLOduration=2.081243066 podStartE2EDuration="2.081243066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:28:45.078087167 +0000 UTC m=+1979.017788500" watchObservedRunningTime="2026-09-11 13:28:45.081243066 +0000 UTC m=+1979.020944399"
Sep 11 13:28:56.064854 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:28:56.064821    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqmc7"
Sep 11 13:30:40.059948 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.059890    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"]
Sep 11 13:30:40.064388 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.064362    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:40.073400 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.072799    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"]
Sep 11 13:30:40.202392 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.202358    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9485c960-5c85-48df-a53f-4371487cedbb-openshift-service-ca\") pod \"maas-api-fdd88446b-7zqrd\" (UID: \"9485c960-5c85-48df-a53f-4371487cedbb\") " pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:40.202392 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.202392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9485c960-5c85-48df-a53f-4371487cedbb-maas-api-tls\") pod \"maas-api-fdd88446b-7zqrd\" (UID: \"9485c960-5c85-48df-a53f-4371487cedbb\") " pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:40.202631 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.202442    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9485c960-5c85-48df-a53f-4371487cedbb-metrics-certs\") pod \"maas-api-fdd88446b-7zqrd\" (UID: \"9485c960-5c85-48df-a53f-4371487cedbb\") " pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:40.202631 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.202507    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nvz7\" (UniqueName: \"kubernetes.io/projected/9485c960-5c85-48df-a53f-4371487cedbb-kube-api-access-9nvz7\") pod \"maas-api-fdd88446b-7zqrd\" (UID: \"9485c960-5c85-48df-a53f-4371487cedbb\") " pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:40.303598 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.303553    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nvz7\" (UniqueName: \"kubernetes.io/projected/9485c960-5c85-48df-a53f-4371487cedbb-kube-api-access-9nvz7\") pod \"maas-api-fdd88446b-7zqrd\" (UID: \"9485c960-5c85-48df-a53f-4371487cedbb\") " pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:40.303801 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.303638    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9485c960-5c85-48df-a53f-4371487cedbb-openshift-service-ca\") pod \"maas-api-fdd88446b-7zqrd\" (UID: \"9485c960-5c85-48df-a53f-4371487cedbb\") " pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:40.303801 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.303656    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9485c960-5c85-48df-a53f-4371487cedbb-maas-api-tls\") pod \"maas-api-fdd88446b-7zqrd\" (UID: \"9485c960-5c85-48df-a53f-4371487cedbb\") " pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:40.303801 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.303691    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9485c960-5c85-48df-a53f-4371487cedbb-metrics-certs\") pod \"maas-api-fdd88446b-7zqrd\" (UID: \"9485c960-5c85-48df-a53f-4371487cedbb\") " pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:40.304359 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.304326    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9485c960-5c85-48df-a53f-4371487cedbb-openshift-service-ca\") pod \"maas-api-fdd88446b-7zqrd\" (UID: \"9485c960-5c85-48df-a53f-4371487cedbb\") " pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:40.306126 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.306101    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9485c960-5c85-48df-a53f-4371487cedbb-maas-api-tls\") pod \"maas-api-fdd88446b-7zqrd\" (UID: \"9485c960-5c85-48df-a53f-4371487cedbb\") " pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:40.306235 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.306217    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9485c960-5c85-48df-a53f-4371487cedbb-metrics-certs\") pod \"maas-api-fdd88446b-7zqrd\" (UID: \"9485c960-5c85-48df-a53f-4371487cedbb\") " pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:40.314656 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.314589    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nvz7\" (UniqueName: \"kubernetes.io/projected/9485c960-5c85-48df-a53f-4371487cedbb-kube-api-access-9nvz7\") pod \"maas-api-fdd88446b-7zqrd\" (UID: \"9485c960-5c85-48df-a53f-4371487cedbb\") " pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:40.378169 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.378129    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:40.659496 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:40.659466    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"]
Sep 11 13:30:41.652958 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:41.652922    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd" event={"ID":"9485c960-5c85-48df-a53f-4371487cedbb","Type":"ContainerStarted","Data":"acfe8fec13edd3c3b8cb930f3061d9639ee1b1f04d79f06c33f5e07d52003603"}
Sep 11 13:30:41.653400 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:41.652987    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd" event={"ID":"9485c960-5c85-48df-a53f-4371487cedbb","Type":"ContainerStarted","Data":"e25ed8b1dc7338e059536e60a98e20aec0df7b038af1ccc1f4d10203b1941b36"}
Sep 11 13:30:41.653400 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:41.653128    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:41.678481 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:41.678405    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd" podStartSLOduration=1.164851857 podStartE2EDuration="1.678383173s" totalImagesPullingTime="513.531316ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:30:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:30:41.677851399 +0000 UTC m=+2095.617552734" watchObservedRunningTime="2026-09-11 13:30:41.678383173 +0000 UTC m=+2095.618084508"
Sep 11 13:30:47.664781 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:47.664747    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-fdd88446b-7zqrd"
Sep 11 13:30:56.051277 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:56.051236    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-78554b866-tht9f"]
Sep 11 13:30:56.051902 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:56.051615    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f" podUID="fefb898b-d767-4ab3-8b74-b8210565ffa4" containerName="maas-api" containerID="cri-o://83eaa654ed2d6d2b67c987fee655a8bbddffa087b7cec126a3e94baa65edaf72" gracePeriod=30
Sep 11 13:30:57.313270 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.313239    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:30:57.376723 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.376630    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fefb898b-d767-4ab3-8b74-b8210565ffa4-maas-api-tls\") pod \"fefb898b-d767-4ab3-8b74-b8210565ffa4\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") "
Sep 11 13:30:57.376723 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.376684    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fefb898b-d767-4ab3-8b74-b8210565ffa4-openshift-service-ca\") pod \"fefb898b-d767-4ab3-8b74-b8210565ffa4\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") "
Sep 11 13:30:57.376958 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.376735    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb898b-d767-4ab3-8b74-b8210565ffa4-metrics-certs\") pod \"fefb898b-d767-4ab3-8b74-b8210565ffa4\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") "
Sep 11 13:30:57.376958 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.376779    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m88wt\" (UniqueName: \"kubernetes.io/projected/fefb898b-d767-4ab3-8b74-b8210565ffa4-kube-api-access-m88wt\") pod \"fefb898b-d767-4ab3-8b74-b8210565ffa4\" (UID: \"fefb898b-d767-4ab3-8b74-b8210565ffa4\") "
Sep 11 13:30:57.377063 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.377028    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fefb898b-d767-4ab3-8b74-b8210565ffa4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fefb898b-d767-4ab3-8b74-b8210565ffa4" (UID: "fefb898b-d767-4ab3-8b74-b8210565ffa4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:30:57.378910 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.378863    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fefb898b-d767-4ab3-8b74-b8210565ffa4-kube-api-access-m88wt" (OuterVolumeSpecName: "kube-api-access-m88wt") pod "fefb898b-d767-4ab3-8b74-b8210565ffa4" (UID: "fefb898b-d767-4ab3-8b74-b8210565ffa4"). InnerVolumeSpecName "kube-api-access-m88wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:30:57.378910 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.378890    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fefb898b-d767-4ab3-8b74-b8210565ffa4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fefb898b-d767-4ab3-8b74-b8210565ffa4" (UID: "fefb898b-d767-4ab3-8b74-b8210565ffa4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:30:57.379047 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.378959    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fefb898b-d767-4ab3-8b74-b8210565ffa4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fefb898b-d767-4ab3-8b74-b8210565ffa4" (UID: "fefb898b-d767-4ab3-8b74-b8210565ffa4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:30:57.478289 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.478195    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m88wt\" (UniqueName: \"kubernetes.io/projected/fefb898b-d767-4ab3-8b74-b8210565ffa4-kube-api-access-m88wt\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:30:57.478289 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.478232    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fefb898b-d767-4ab3-8b74-b8210565ffa4-maas-api-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:30:57.478289 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.478246    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fefb898b-d767-4ab3-8b74-b8210565ffa4-openshift-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:30:57.478289 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.478260    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fefb898b-d767-4ab3-8b74-b8210565ffa4-metrics-certs\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\""
Sep 11 13:30:57.736876 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.736838    2582 generic.go:356] "Generic (PLEG): container finished" podID="fefb898b-d767-4ab3-8b74-b8210565ffa4" containerID="83eaa654ed2d6d2b67c987fee655a8bbddffa087b7cec126a3e94baa65edaf72" exitCode=0
Sep 11 13:30:57.737046 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.736884    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f"
Sep 11 13:30:57.737046 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.736918    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f" event={"ID":"fefb898b-d767-4ab3-8b74-b8210565ffa4","Type":"ContainerDied","Data":"83eaa654ed2d6d2b67c987fee655a8bbddffa087b7cec126a3e94baa65edaf72"}
Sep 11 13:30:57.737046 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.736975    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78554b866-tht9f" event={"ID":"fefb898b-d767-4ab3-8b74-b8210565ffa4","Type":"ContainerDied","Data":"a6065ee10138a8e75d8a95614557f81166cfd4d286376a0f09676b9dcb0cca4b"}
Sep 11 13:30:57.737046 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.736997    2582 scope.go:117] "RemoveContainer" containerID="83eaa654ed2d6d2b67c987fee655a8bbddffa087b7cec126a3e94baa65edaf72"
Sep 11 13:30:57.746337 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.746320    2582 scope.go:117] "RemoveContainer" containerID="83eaa654ed2d6d2b67c987fee655a8bbddffa087b7cec126a3e94baa65edaf72"
Sep 11 13:30:57.746629 ip-10-0-143-6 kubenswrapper[2582]: E0911 13:30:57.746609    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83eaa654ed2d6d2b67c987fee655a8bbddffa087b7cec126a3e94baa65edaf72\": container with ID starting with 83eaa654ed2d6d2b67c987fee655a8bbddffa087b7cec126a3e94baa65edaf72 not found: ID does not exist" containerID="83eaa654ed2d6d2b67c987fee655a8bbddffa087b7cec126a3e94baa65edaf72"
Sep 11 13:30:57.746700 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.746637    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83eaa654ed2d6d2b67c987fee655a8bbddffa087b7cec126a3e94baa65edaf72"} err="failed to get container status \"83eaa654ed2d6d2b67c987fee655a8bbddffa087b7cec126a3e94baa65edaf72\": rpc error: code = NotFound desc = could not find container \"83eaa654ed2d6d2b67c987fee655a8bbddffa087b7cec126a3e94baa65edaf72\": container with ID starting with 83eaa654ed2d6d2b67c987fee655a8bbddffa087b7cec126a3e94baa65edaf72 not found: ID does not exist"
Sep 11 13:30:57.759736 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.759710    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-78554b866-tht9f"]
Sep 11 13:30:57.767735 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:57.767713    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-78554b866-tht9f"]
Sep 11 13:30:58.477667 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:30:58.477624    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fefb898b-d767-4ab3-8b74-b8210565ffa4" path="/var/lib/kubelet/pods/fefb898b-d767-4ab3-8b74-b8210565ffa4/volumes"
Sep 11 13:31:12.218400 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:12.218362    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-679f8bfbf-7v77p_b613ea7b-3d4f-41cd-8c03-2d720452a603/authorino/0.log"
Sep 11 13:31:17.797153 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:17.797122    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-h2mb4_c584055b-afaa-4aae-8c85-821b41963def/manager/0.log"
Sep 11 13:31:17.927779 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:17.927742    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-z2mzt_5fe1552b-1a45-4a6c-83a6-145ed7f3650f/manager/0.log"
Sep 11 13:31:18.053447 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:18.053354    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-65b89bb8c4-ktsgf_d8e53bd3-2211-4c00-8fea-5726fe325da8/manager/0.log"
Sep 11 13:31:18.171671 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:18.171639    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bkmhq_c3924b26-63e9-459e-8f20-4e516beffdac/server/0.log"
Sep 11 13:31:18.301693 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:18.301655    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-v6gnm_d9e643ac-fe88-4914-84ca-226e2d9a2059/manager/0.log"
Sep 11 13:31:18.421786 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:18.421700    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f55dc566-2l7wq_55a86612-fe83-4731-a2df-1e5c9e79c1fb/manager/0.log"
Sep 11 13:31:18.428069 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:18.428050    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f55dc566-2l7wq_55a86612-fe83-4731-a2df-1e5c9e79c1fb/copy-manifests/0.log"
Sep 11 13:31:19.054990 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:19.054945    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-fdd88446b-7zqrd_9485c960-5c85-48df-a53f-4371487cedbb/maas-api/0.log"
Sep 11 13:31:19.517914 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:19.517879    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-8sm8f_36f77989-5116-42ab-832e-ee5fc4eb42ac/postgres/0.log"
Sep 11 13:31:20.327708 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:20.327662    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-679f8bfbf-7v77p_b613ea7b-3d4f-41cd-8c03-2d720452a603/authorino/0.log"
Sep 11 13:31:20.836919 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:20.836879    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-zqmc7_7de30b0f-896e-4b4c-a96c-d5468982c05a/manager/0.log"
Sep 11 13:31:22.141898 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:22.141866    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jhml6_fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90/payload-pre-processing/0.log"
Sep 11 13:31:22.271619 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:22.271563    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-84qbs_71f3fe4f-d604-49ce-8364-6643e5c1afc1/payload-processing/0.log"
Sep 11 13:31:22.760863 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:22.760796    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-9pmtf_f9e446a0-924c-4a31-be2a-266d861fe6c3/main/0.log"
Sep 11 13:31:23.114118 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:23.114024    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-kr62x_18b2ad70-2bd1-483c-9c53-2e5c329b6bb3/main/0.log"
Sep 11 13:31:32.518914 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:32.518879    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lbn4f_da6d1a19-06a1-4a7a-bbc2-593b438eeb41/global-pull-secret-syncer/0.log"
Sep 11 13:31:32.824193 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:32.824105    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zkjt7_b3d2a78d-a895-4b2f-b965-bd1cb39bdff2/konnectivity-agent/0.log"
Sep 11 13:31:33.034952 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:33.034915    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-6.ec2.internal_bf433a3f8ec5b3534116bb587dadd6f1/haproxy/0.log"
Sep 11 13:31:37.077313 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:37.077276    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-679f8bfbf-7v77p_b613ea7b-3d4f-41cd-8c03-2d720452a603/authorino/0.log"
Sep 11 13:31:37.267343 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:37.267306    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-zqmc7_7de30b0f-896e-4b4c-a96c-d5468982c05a/manager/0.log"
Sep 11 13:31:38.863171 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:38.863135    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0a2a14b2-8b2e-4647-bac0-0346c745e357/alertmanager/0.log"
Sep 11 13:31:38.891627 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:38.891548    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0a2a14b2-8b2e-4647-bac0-0346c745e357/config-reloader/0.log"
Sep 11 13:31:38.921455 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:38.921405    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0a2a14b2-8b2e-4647-bac0-0346c745e357/kube-rbac-proxy-web/0.log"
Sep 11 13:31:38.944560 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:38.944534    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0a2a14b2-8b2e-4647-bac0-0346c745e357/kube-rbac-proxy/0.log"
Sep 11 13:31:38.968629 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:38.968597    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0a2a14b2-8b2e-4647-bac0-0346c745e357/kube-rbac-proxy-metric/0.log"
Sep 11 13:31:38.993723 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:38.993693    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0a2a14b2-8b2e-4647-bac0-0346c745e357/prom-label-proxy/0.log"
Sep 11 13:31:39.019544 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.019514    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0a2a14b2-8b2e-4647-bac0-0346c745e357/init-config-reloader/0.log"
Sep 11 13:31:39.215068 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.215029    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7875bc4d55-kpt8w_0453471e-decd-4d92-9a1d-2af871b1aac6/metrics-server/0.log"
Sep 11 13:31:39.407103 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.407070    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ls8jx_fe281be4-697f-4ff8-8ab7-153c0987e10d/node-exporter/0.log"
Sep 11 13:31:39.428253 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.428157    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ls8jx_fe281be4-697f-4ff8-8ab7-153c0987e10d/kube-rbac-proxy/0.log"
Sep 11 13:31:39.451956 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.451931    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ls8jx_fe281be4-697f-4ff8-8ab7-153c0987e10d/init-textfile/0.log"
Sep 11 13:31:39.590070 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.590037    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-7xl7v_439f4bb3-3bc7-40f9-8c92-9110801fc1f1/kube-rbac-proxy-main/0.log"
Sep 11 13:31:39.615666 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.615602    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-7xl7v_439f4bb3-3bc7-40f9-8c92-9110801fc1f1/kube-rbac-proxy-self/0.log"
Sep 11 13:31:39.650548 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.650517    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-7xl7v_439f4bb3-3bc7-40f9-8c92-9110801fc1f1/openshift-state-metrics/0.log"
Sep 11 13:31:39.695726 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.695638    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbaa8dc3-cb17-47e6-9b02-5499e3694961/prometheus/0.log"
Sep 11 13:31:39.720516 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.720468    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbaa8dc3-cb17-47e6-9b02-5499e3694961/config-reloader/0.log"
Sep 11 13:31:39.749020 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.748989    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbaa8dc3-cb17-47e6-9b02-5499e3694961/thanos-sidecar/0.log"
Sep 11 13:31:39.782070 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.782035    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbaa8dc3-cb17-47e6-9b02-5499e3694961/kube-rbac-proxy-web/0.log"
Sep 11 13:31:39.816864 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.816835    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbaa8dc3-cb17-47e6-9b02-5499e3694961/kube-rbac-proxy/0.log"
Sep 11 13:31:39.845458 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.845413    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbaa8dc3-cb17-47e6-9b02-5499e3694961/kube-rbac-proxy-thanos/0.log"
Sep 11 13:31:39.872142 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.872121    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbaa8dc3-cb17-47e6-9b02-5499e3694961/init-config-reloader/0.log"
Sep 11 13:31:39.908064 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.908038    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-hdbdn_78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d/prometheus-operator/0.log"
Sep 11 13:31:39.931656 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.931629    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-hdbdn_78e0c53c-3e9a-46aa-9f67-ee1c8fa6577d/kube-rbac-proxy/0.log"
Sep 11 13:31:39.965324 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:39.965248    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-42vpt_bc85ed6c-96aa-4ca5-839e-073142c4b699/prometheus-operator-admission-webhook/0.log"
Sep 11 13:31:40.125875 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:40.125840    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-bb957f59b-2nsgs_126a62b4-8979-4c34-a839-bb18421a90a7/thanos-query/0.log"
Sep 11 13:31:40.153270 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:40.153239    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-bb957f59b-2nsgs_126a62b4-8979-4c34-a839-bb18421a90a7/kube-rbac-proxy-web/0.log"
Sep 11 13:31:40.181496 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:40.181460    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-bb957f59b-2nsgs_126a62b4-8979-4c34-a839-bb18421a90a7/kube-rbac-proxy/0.log"
Sep 11 13:31:40.208216 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:40.208189    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-bb957f59b-2nsgs_126a62b4-8979-4c34-a839-bb18421a90a7/prom-label-proxy/0.log"
Sep 11 13:31:40.239407 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:40.239382    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-bb957f59b-2nsgs_126a62b4-8979-4c34-a839-bb18421a90a7/kube-rbac-proxy-rules/0.log"
Sep 11 13:31:40.278480 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:40.278450    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-bb957f59b-2nsgs_126a62b4-8979-4c34-a839-bb18421a90a7/kube-rbac-proxy-metrics/0.log"
Sep 11 13:31:42.028823 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.028772    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-9d9zz_ba03dab9-0722-48d6-9441-87eacb436c12/networking-console-plugin/0.log"
Sep 11 13:31:42.740555 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.740523    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"]
Sep 11 13:31:42.740984 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.740960    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fefb898b-d767-4ab3-8b74-b8210565ffa4" containerName="maas-api"
Sep 11 13:31:42.740984 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.740977    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fefb898b-d767-4ab3-8b74-b8210565ffa4" containerName="maas-api"
Sep 11 13:31:42.741187 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.741064    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fefb898b-d767-4ab3-8b74-b8210565ffa4" containerName="maas-api"
Sep 11 13:31:42.744565 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.744547    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:42.747445 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.747407    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ljsx5\"/\"kube-root-ca.crt\""
Sep 11 13:31:42.748566 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.748549    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ljsx5\"/\"openshift-service-ca.crt\""
Sep 11 13:31:42.748647 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.748627    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ljsx5\"/\"default-dockercfg-cht8v\""
Sep 11 13:31:42.757393 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.757364    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"]
Sep 11 13:31:42.809702 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.809659    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xwhd\" (UniqueName: \"kubernetes.io/projected/c5a567bf-29d4-4f9e-9fb2-af6e5593fee1-kube-api-access-8xwhd\") pod \"perf-node-gather-daemonset-mp8mg\" (UID: \"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1\") " pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:42.809863 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.809708    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5a567bf-29d4-4f9e-9fb2-af6e5593fee1-sys\") pod \"perf-node-gather-daemonset-mp8mg\" (UID: \"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1\") " pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:42.809863 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.809730    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c5a567bf-29d4-4f9e-9fb2-af6e5593fee1-podres\") pod \"perf-node-gather-daemonset-mp8mg\" (UID: \"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1\") " pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:42.809863 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.809817    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5a567bf-29d4-4f9e-9fb2-af6e5593fee1-lib-modules\") pod \"perf-node-gather-daemonset-mp8mg\" (UID: \"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1\") " pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:42.809968 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.809936    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c5a567bf-29d4-4f9e-9fb2-af6e5593fee1-proc\") pod \"perf-node-gather-daemonset-mp8mg\" (UID: \"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1\") " pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:42.910705 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.910662    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c5a567bf-29d4-4f9e-9fb2-af6e5593fee1-proc\") pod \"perf-node-gather-daemonset-mp8mg\" (UID: \"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1\") " pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:42.910885 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.910726    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xwhd\" (UniqueName: \"kubernetes.io/projected/c5a567bf-29d4-4f9e-9fb2-af6e5593fee1-kube-api-access-8xwhd\") pod \"perf-node-gather-daemonset-mp8mg\" (UID: \"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1\") " pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:42.910885 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.910755    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5a567bf-29d4-4f9e-9fb2-af6e5593fee1-sys\") pod \"perf-node-gather-daemonset-mp8mg\" (UID: \"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1\") " pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:42.910885 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.910781    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c5a567bf-29d4-4f9e-9fb2-af6e5593fee1-podres\") pod \"perf-node-gather-daemonset-mp8mg\" (UID: \"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1\") " pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:42.910885 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.910810    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5a567bf-29d4-4f9e-9fb2-af6e5593fee1-sys\") pod \"perf-node-gather-daemonset-mp8mg\" (UID: \"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1\") " pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:42.910885 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.910781    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c5a567bf-29d4-4f9e-9fb2-af6e5593fee1-proc\") pod \"perf-node-gather-daemonset-mp8mg\" (UID: \"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1\") " pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:42.910885 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.910854    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5a567bf-29d4-4f9e-9fb2-af6e5593fee1-lib-modules\") pod \"perf-node-gather-daemonset-mp8mg\" (UID: \"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1\") " pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:42.911116 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.910909    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c5a567bf-29d4-4f9e-9fb2-af6e5593fee1-podres\") pod \"perf-node-gather-daemonset-mp8mg\" (UID: \"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1\") " pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:42.911116 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.911002    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5a567bf-29d4-4f9e-9fb2-af6e5593fee1-lib-modules\") pod \"perf-node-gather-daemonset-mp8mg\" (UID: \"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1\") " pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:42.919709 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:42.919685    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xwhd\" (UniqueName: \"kubernetes.io/projected/c5a567bf-29d4-4f9e-9fb2-af6e5593fee1-kube-api-access-8xwhd\") pod \"perf-node-gather-daemonset-mp8mg\" (UID: \"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1\") " pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:43.058092 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:43.058011    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:43.333295 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:43.333261    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"]
Sep 11 13:31:43.334766 ip-10-0-143-6 kubenswrapper[2582]: W0911 13:31:43.334733    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc5a567bf_29d4_4f9e_9fb2_af6e5593fee1.slice/crio-dffb55a06d7ab23be02f2ec0d92337016dd1fcdeba63784a56ab93f4453bf2b8 WatchSource:0}: Error finding container dffb55a06d7ab23be02f2ec0d92337016dd1fcdeba63784a56ab93f4453bf2b8: Status 404 returned error can't find the container with id dffb55a06d7ab23be02f2ec0d92337016dd1fcdeba63784a56ab93f4453bf2b8
Sep 11 13:31:43.382862 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:43.382838    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-2zvkx_7a54332c-5c8c-4224-ad6b-e9e4be156923/download-server/0.log"
Sep 11 13:31:43.976973 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:43.976938    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg" event={"ID":"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1","Type":"ContainerStarted","Data":"13939124fc00028963f79897b0cd3cb972b194dbf4d3195b8eebe9faa415d369"}
Sep 11 13:31:43.977146 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:43.976989    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg" event={"ID":"c5a567bf-29d4-4f9e-9fb2-af6e5593fee1","Type":"ContainerStarted","Data":"dffb55a06d7ab23be02f2ec0d92337016dd1fcdeba63784a56ab93f4453bf2b8"}
Sep 11 13:31:43.977146 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:43.977068    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:44.001986 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:44.001938    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg" podStartSLOduration=2.001923906 podStartE2EDuration="2.001923906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:31:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:31:44.000765905 +0000 UTC m=+2157.940467261" watchObservedRunningTime="2026-09-11 13:31:44.001923906 +0000 UTC m=+2157.941625276"
Sep 11 13:31:45.611039 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:45.611008    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nt9jt_60ebfb6b-c42a-4634-ad12-24056202391e/dns/0.log"
Sep 11 13:31:45.644440 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:45.644393    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nt9jt_60ebfb6b-c42a-4634-ad12-24056202391e/kube-rbac-proxy/0.log"
Sep 11 13:31:45.683850 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:45.683816    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8lsb4_8c08440e-84a0-40a3-b171-92220128ee4c/dns-node-resolver/0.log"
Sep 11 13:31:46.357196 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:46.357159    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5f7b4f7974-hwkmf_21a85115-2aac-4af5-b310-f6334c7ecaa5/registry/0.log"
Sep 11 13:31:46.423714 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:46.423683    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sm29q_b4af872d-253e-474d-ada0-07f8ac808485/node-ca/0.log"
Sep 11 13:31:48.412011 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:48.411969    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jhml6_fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90/payload-pre-processing/0.log"
Sep 11 13:31:48.457942 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:48.457905    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-84qbs_71f3fe4f-d604-49ce-8364-6643e5c1afc1/payload-processing/0.log"
Sep 11 13:31:49.655601 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:49.655574    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vf4j6_c96a7bc4-bd26-4324-b364-2bc0b5f2ea9d/serve-healthcheck-canary/0.log"
Sep 11 13:31:49.992645 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:49.992615    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ljsx5/perf-node-gather-daemonset-mp8mg"
Sep 11 13:31:50.292293 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:50.292207    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fvrvd_0299ca51-7147-46ea-83f9-90e9efca03c7/kube-rbac-proxy/0.log"
Sep 11 13:31:50.321944 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:50.321912    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fvrvd_0299ca51-7147-46ea-83f9-90e9efca03c7/exporter/0.log"
Sep 11 13:31:50.348706 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:50.348677    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fvrvd_0299ca51-7147-46ea-83f9-90e9efca03c7/extractor/0.log"
Sep 11 13:31:52.989356 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:52.989310    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-h2mb4_c584055b-afaa-4aae-8c85-821b41963def/manager/0.log"
Sep 11 13:31:53.314139 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:53.314054    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-z2mzt_5fe1552b-1a45-4a6c-83a6-145ed7f3650f/manager/0.log"
Sep 11 13:31:53.385226 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:53.385180    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-65b89bb8c4-ktsgf_d8e53bd3-2211-4c00-8fea-5726fe325da8/manager/0.log"
Sep 11 13:31:53.415562 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:53.415532    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bkmhq_c3924b26-63e9-459e-8f20-4e516beffdac/server/0.log"
Sep 11 13:31:53.551845 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:53.551810    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-v6gnm_d9e643ac-fe88-4914-84ca-226e2d9a2059/manager/0.log"
Sep 11 13:31:53.596912 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:53.596822    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f55dc566-2l7wq_55a86612-fe83-4731-a2df-1e5c9e79c1fb/manager/0.log"
Sep 11 13:31:53.637340 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:53.637312    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56f55dc566-2l7wq_55a86612-fe83-4731-a2df-1e5c9e79c1fb/copy-manifests/0.log"
Sep 11 13:31:54.518966 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:54.518937    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jhml6_fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90/payload-pre-processing/0.log"
Sep 11 13:31:54.558854 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:54.558816    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-84qbs_71f3fe4f-d604-49ce-8364-6643e5c1afc1/payload-processing/0.log"
Sep 11 13:31:55.233494 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:55.233462    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5cd99cffb8-82pss_786f3263-387a-4421-95ea-4bc7555a3b55/manager/0.log"
Sep 11 13:31:55.305341 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:55.305307    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-ht9ww_4ad5ab77-3c1c-444e-8007-4aed0811f02b/openshift-lws-operator/0.log"
Sep 11 13:31:56.838737 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:56.838702    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jhml6_fa1cd2d7-fe47-45f6-b0ca-e36fb1c44d90/payload-pre-processing/0.log"
Sep 11 13:31:56.884869 ip-10-0-143-6 kubenswrapper[2582]: I0911 13:31:56.884835    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-84qbs_71f3fe4f-d604-49ce-8364-6643e5c1afc1/payload-processing/0.log"