Sep 19 11:32:16.707571 ip-10-0-140-136 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 19 11:32:16.707581 ip-10-0-140-136 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 19 11:32:16.707591 ip-10-0-140-136 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 19 11:32:16.707770 ip-10-0-140-136 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 19 11:32:26.807215 ip-10-0-140-136 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 19 11:32:26.807236 ip-10-0-140-136 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot ddd79ca5e7874bd2bae20dc6599c9244 --
Sep 19 11:34:06.764465 ip-10-0-140-136 systemd[1]: Starting Kubernetes Kubelet...
Sep 19 11:34:07.160100 ip-10-0-140-136 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 11:34:07.160100 ip-10-0-140-136 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 19 11:34:07.160100 ip-10-0-140-136 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 11:34:07.160100 ip-10-0-140-136 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 19 11:34:07.160100 ip-10-0-140-136 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 11:34:07.161221 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.161098    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 19 11:34:07.164383 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164368    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 11:34:07.164383 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164384    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164387    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164391    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164393    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164397    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164407    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164410    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164413    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164416    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164419    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164422    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164425    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164428    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164430    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164433    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164436    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164438    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164440    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164443    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164446    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 11:34:07.164444 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164448    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164451    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164454    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164456    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164460    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164464    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164467    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164470    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164472    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164475    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164477    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164479    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164482    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164484    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164487    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164489    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164492    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164495    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164497    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 11:34:07.164927 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164500    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164502    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164505    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164507    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164509    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164512    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164514    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164517    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164519    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164522    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164524    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164527    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164529    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164532    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164536    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164539    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164542    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164545    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164549    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 11:34:07.165438 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164553    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164556    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164559    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164563    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164566    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164568    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164571    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164574    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164577    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164580    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164582    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164585    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164588    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164590    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164592    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164595    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164597    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164600    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164603    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 11:34:07.165905 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164605    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164608    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164610    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164613    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164615    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164618    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164620    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164623    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164625    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164629    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164631    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164634    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164636    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164639    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164641    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164643    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164646    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164648    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164651    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164653    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 11:34:07.166379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.164655    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165336    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165343    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165346    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165349    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165352    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165355    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165358    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165361    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165363    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165366    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165371    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165375    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165378    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165380    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165383    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165386    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165388    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165391    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165393    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 11:34:07.166859 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165396    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165398    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165401    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165404    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165408    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165412    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165415    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165417    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165427    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165430    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165433    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165436    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165438    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165441    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165444    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165447    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165451    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165454    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165457    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 11:34:07.167379 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165461    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165463    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165466    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165469    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165472    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165474    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165477    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165480    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165482    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165485    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165487    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165489    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165492    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165494    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165497    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165499    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165502    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165504    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165507    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165511    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 11:34:07.167848 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165513    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165515    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165518    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165520    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165523    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165525    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165527    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165530    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165532    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165536    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165538    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165541    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165544    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165548    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165550    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165553    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165556    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165558    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165562    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 11:34:07.168361 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165564    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165567    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165569    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165572    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165574    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165577    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165581    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165584    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165587    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165589    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165592    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165594    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165597    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165599    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165602    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165604    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165607    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165609    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165611    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165614    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 11:34:07.168833 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165616    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165619    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165987    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165992    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165995    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.165998    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166006    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166009    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166012    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166015    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166017    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166020    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166023    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166025    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166028    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166030    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166033    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166035    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166038    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166040    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 11:34:07.169362 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166043    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166045    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166048    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166050    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166053    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166055    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166058    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166060    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166062    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166065    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166067    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166070    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166072    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166076    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166080    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166082    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166085    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166089    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166093    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 11:34:07.169857 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166101    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166104    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166107    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166110    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166113    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166115    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166117    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166120    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166122    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166125    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166128    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166130    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166132    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166135    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166137    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166140    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166142    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166145    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166147    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166150    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 11:34:07.170350 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166152    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166154    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166157    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166159    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166161    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166164    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166167    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166170    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166172    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166175    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166178    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166196    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166200    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166210    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166213    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166215    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166218    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166220    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166222    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166225    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 11:34:07.170880 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166228    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166230    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166233    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166235    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166238    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166240    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166243    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166245    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166248    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166250    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166252    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166255    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166257    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166261    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166265    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166268    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166270    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166273    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166279    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 11:34:07.171372 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166282    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166285    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.166287    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166364    2559 flags.go:64] FLAG: --address="0.0.0.0"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166396    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166402    2559 flags.go:64] FLAG: --anonymous-auth="true"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166406    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166416    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166419    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166423    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166427    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166430    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166433    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166436    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166439    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166442    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166445    2559 flags.go:64] FLAG: --cgroup-root=""
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166448    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166450    2559 flags.go:64] FLAG: --client-ca-file=""
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166453    2559 flags.go:64] FLAG: --cloud-provider="external"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166456    2559 flags.go:64] FLAG: --cluster-dns="[]"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166465    2559 flags.go:64] FLAG: --cluster-domain=""
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166468    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166470    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 19 11:34:07.171836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166473    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166477    2559 flags.go:64] FLAG: --container-log-max-files="5"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166481    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166484    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166486    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166490    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166493    2559 flags.go:64] FLAG: --contention-profiling="false"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166497    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166500    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166503    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166506    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166514    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166516    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166519    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166522    2559 flags.go:64] FLAG: --enable-load-reader="false"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166525    2559 flags.go:64] FLAG: --enable-server="true"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166534    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166541    2559 flags.go:64] FLAG: --event-burst="100"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166544    2559 flags.go:64] FLAG: --event-qps="50"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166547    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166549    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166552    2559 flags.go:64] FLAG: --eviction-hard=""
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166556    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166559    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166562    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 19 11:34:07.172472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166564    2559 flags.go:64] FLAG: --eviction-soft=""
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166567    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166570    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166573    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166575    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166578    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166581    2559 flags.go:64] FLAG: --fail-swap-on="true"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166584    2559 flags.go:64] FLAG: --feature-gates=""
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166587    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166590    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166593    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166596    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166599    2559 flags.go:64] FLAG: --healthz-port="10248"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166602    2559 flags.go:64] FLAG: --help="false"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166607    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166610    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166613    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166623    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166627    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166630    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166633    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166635    2559 flags.go:64] FLAG: --image-service-endpoint=""
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166638    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 19 11:34:07.173072 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166641    2559 flags.go:64] FLAG: --kube-api-burst="100"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166649    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166652    2559 flags.go:64] FLAG: --kube-api-qps="50"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166655    2559 flags.go:64] FLAG: --kube-reserved=""
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166658    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166660    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166663    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166666    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166668    2559 flags.go:64] FLAG: --lock-file=""
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166671    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166674    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166677    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166682    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166685    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166687    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166690    2559 flags.go:64] FLAG: --logging-format="text"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166693    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166696    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166699    2559 flags.go:64] FLAG: --manifest-url=""
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166701    2559 flags.go:64] FLAG: --manifest-url-header=""
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166705    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166708    2559 flags.go:64] FLAG: --max-open-files="1000000"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166712    2559 flags.go:64] FLAG: --max-pods="110"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166717    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166720    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 19 11:34:07.173681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166723    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166725    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166728    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166731    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166734    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166741    2559 flags.go:64] FLAG: --node-status-max-images="50"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166744    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166746    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166749    2559 flags.go:64] FLAG: --pod-cidr=""
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166758    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166763    2559 flags.go:64] FLAG: --pod-manifest-path=""
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166765    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166768    2559 flags.go:64] FLAG: --pods-per-core="0"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166771    2559 flags.go:64] FLAG: --port="10250"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166774    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166777    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e0d3771c299ceaae"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166780    2559 flags.go:64] FLAG: --qos-reserved=""
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166783    2559 flags.go:64] FLAG: --read-only-port="10255"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166786    2559 flags.go:64] FLAG: --register-node="true"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166788    2559 flags.go:64] FLAG: --register-with-taints=""
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166792    2559 flags.go:64] FLAG: --registry-burst="10"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166795    2559 flags.go:64] FLAG: --registry-qps="5"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166798    2559 flags.go:64] FLAG: --reserved-cpus=""
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166801    2559 flags.go:64] FLAG: --reserved-memory=""
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166804    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 19 11:34:07.174996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166807    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166810    2559 flags.go:64] FLAG: --rotate-certificates="false"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166813    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166815    2559 flags.go:64] FLAG: --runonce="false"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166818    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166822    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166826    2559 flags.go:64] FLAG: --seccomp-default="false"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166829    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166832    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166835    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166838    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166841    2559 flags.go:64] FLAG: --storage-driver-password="root"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166844    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166847    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166849    2559 flags.go:64] FLAG: --storage-driver-user="root"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166852    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166855    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166863    2559 flags.go:64] FLAG: --system-cgroups=""
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166866    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166871    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166874    2559 flags.go:64] FLAG: --tls-cert-file=""
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166877    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166883    2559 flags.go:64] FLAG: --tls-min-version=""
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166886    2559 flags.go:64] FLAG: --tls-private-key-file=""
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166889    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 19 11:34:07.175738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166892    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166895    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166898    2559 flags.go:64] FLAG: --v="2"
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166901    2559 flags.go:64] FLAG: --version="false"
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166905    2559 flags.go:64] FLAG: --vmodule=""
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166908    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.166911    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167043    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167047    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167050    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167053    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167055    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167059    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167063    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167066    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167068    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167071    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167073    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167076    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167078    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167081    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167083    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 11:34:07.176708 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167086    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167088    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167092    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167101    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167104    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167106    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167108    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167111    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167113    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167116    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167118    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167120    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167123    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167125    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167128    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167131    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167133    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167136    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167138    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 11:34:07.177328 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167142    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167146    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167149    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167153    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167157    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167160    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167162    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167165    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167167    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167170    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167173    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167175    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167178    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167194    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167197    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167200    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167203    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167212    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167215    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 11:34:07.177817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167218    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167221    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167223    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167226    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167228    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167231    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167233    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167236    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167238    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167241    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167244    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167246    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167249    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167251    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167254    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167256    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167260    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167264    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167267    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167270    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 11:34:07.178317 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167272    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167275    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167277    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167280    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167282    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167285    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167287    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167290    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167292    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167294    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167297    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167305    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167308    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167310    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167313    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167315    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167318    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167320    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167322    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167325    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 11:34:07.178807 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167327    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 11:34:07.179321 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167330    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 11:34:07.179321 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167332    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 11:34:07.179321 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167335    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 11:34:07.179321 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167337    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 11:34:07.179321 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.167339    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 11:34:07.179321 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.168048    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 11:34:07.179321 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.177727    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 19 11:34:07.179321 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.177743    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 19 11:34:07.179321 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.178678    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 19 11:34:07.179321 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.178686    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 19 11:34:07.179321 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178742    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 11:34:07.179321 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178748    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 11:34:07.179321 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178751    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 11:34:07.179321 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178754    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178757    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178761    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178764    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178767    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178769    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178773    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178777    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178780    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178783    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178786    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178789    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178793    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178795    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178798    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178801    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178803    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178806    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178808    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178811    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 11:34:07.179674 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178814    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178816    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178819    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178822    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178824    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178827    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178830    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178833    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178836    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178838    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178841    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178843    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178846    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178848    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178851    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178853    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178856    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178859    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178861    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 11:34:07.180165 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178865    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178869    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178872    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178874    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178879    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178882    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178885    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178887    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178890    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178892    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178895    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178897    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178900    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178903    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178905    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178908    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178910    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178912    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178915    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178918    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 11:34:07.180682 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178921    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178923    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178926    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178929    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178932    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178934    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178937    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178939    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178942    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178944    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178947    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178950    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178952    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178954    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178958    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178961    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178964    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178969    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178971    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 11:34:07.181172 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178974    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178977    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178980    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178982    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178985    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178988    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178990    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178993    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178995    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.178998    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179000    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179003    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179005    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179008    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179011    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179013    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179016    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 11:34:07.181661 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179018    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 11:34:07.182069 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.179023    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 11:34:07.182069 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179136    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 11:34:07.182069 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179142    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 11:34:07.182069 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179146    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 11:34:07.182069 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179149    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 11:34:07.182069 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179152    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 11:34:07.182069 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179155    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 11:34:07.182069 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179158    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 11:34:07.182069 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179161    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 11:34:07.182069 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179165    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 11:34:07.182069 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179168    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 11:34:07.182069 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179171    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 11:34:07.182069 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179174    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 11:34:07.182069 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179177    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179180    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179199    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179202    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179205    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179208    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179210    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179213    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179215    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179218    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179220    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179223    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179226    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179228    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179231    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179233    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179236    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179238    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179241    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179243    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 11:34:07.182441 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179246    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179248    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179251    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179253    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179255    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179258    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179260    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179263    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179265    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179268    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179270    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179273    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179275    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179279    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179282    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179285    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179287    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179290    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179292    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179294    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 11:34:07.182924 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179297    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179299    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179302    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179304    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179306    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179309    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179311    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179316    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179319    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179322    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179325    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179327    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179330    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179332    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179335    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179337    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179340    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179343    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179345    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 11:34:07.183423 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179348    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179350    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179353    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179355    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179357    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179360    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179363    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179365    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179368    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179370    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179373    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179375    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179378    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179380    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179383    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179385    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179387    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179390    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179393    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179395    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 11:34:07.183891 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179398    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 11:34:07.184407 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179400    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 11:34:07.184407 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179403    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 11:34:07.184407 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179405    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 11:34:07.184407 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179407    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 11:34:07.184407 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179410    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 11:34:07.184407 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179412    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 11:34:07.184407 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:07.179415    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 11:34:07.184407 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.179420    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 11:34:07.184407 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.179530    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 19 11:34:07.184407 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.183608    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 19 11:34:07.184681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.184669    2559 server.go:1019] "Starting client certificate rotation"
Sep 19 11:34:07.184787 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.184771    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 11:34:07.185463 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.185453    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 11:34:07.187554 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.187541    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 11:34:07.190546 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.190530    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 11:34:07.206654 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.206635    2559 log.go:25] "Validated CRI v1 runtime API"
Sep 19 11:34:07.208616 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.208594    2559 log.go:25] "Validated CRI v1 image API"
Sep 19 11:34:07.209461 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.209445    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 19 11:34:07.212946 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.212928    2559 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b882a34e-97c5-4d9a-ab62-cfdbc265b0bc:/dev/nvme0n1p3 ea3752be-4a79-4d7d-9e11-e70fb674b295:/dev/nvme0n1p4]
Sep 19 11:34:07.213017 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.212945    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 19 11:34:07.213336 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.213320    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 11:34:07.218588 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.218420    2559 manager.go:216] Machine: {Timestamp:2026-09-19 11:34:07.217951561 +0000 UTC m=+0.348269930 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101886 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ea6c904811a14b4889871f785b1d0 SystemUUID:ec2ea6c9-0481-1a14-b488-9871f785b1d0 BootID:ddd79ca5-e787-4bd2-bae2-0dc6599c9244 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:db:1c:40:a3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:db:1c:40:a3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:e9:86:34:33:b0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 19 11:34:07.218588 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.218584    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 19 11:34:07.218690 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.218653    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 19 11:34:07.222046 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.222025    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 19 11:34:07.222173 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.222050    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-136.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 19 11:34:07.223567 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.223556    2559 topology_manager.go:138] "Creating topology manager with none policy"
Sep 19 11:34:07.223606 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.223569    2559 container_manager_linux.go:309] "Creating device plugin manager"
Sep 19 11:34:07.223606 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.223576    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 11:34:07.224346 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.224336    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 11:34:07.226209 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.226199    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 19 11:34:07.231086 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.231055    2559 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 11:34:07.231309 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.231286    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 19 11:34:07.231414 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.231329    2559 kubelet.go:486] "Attempting to sync node with API server"
Sep 19 11:34:07.231414 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.231344    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 19 11:34:07.231414 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.231366    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 19 11:34:07.231414 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.231378    2559 kubelet.go:398] "Adding apiserver pod source"
Sep 19 11:34:07.231414 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.231398    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 19 11:34:07.231984 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.231772    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 11:34:07.231984 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.231797    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 11:34:07.234542 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.234523    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 19 11:34:07.237491 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.237475    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 19 11:34:07.237568 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.237514    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 19 11:34:07.238693 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.238676    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l9ql4"
Sep 19 11:34:07.239085 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.239071    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 19 11:34:07.239155 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.239092    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 19 11:34:07.239155 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.239101    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 19 11:34:07.239155 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.239110    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 19 11:34:07.239155 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.239122    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 19 11:34:07.239155 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.239132    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 19 11:34:07.239155 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.239151    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 19 11:34:07.239386 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.239159    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 19 11:34:07.239386 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.239169    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 19 11:34:07.239386 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.239178    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 19 11:34:07.239386 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.239207    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 19 11:34:07.239386 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.239216    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 19 11:34:07.239386 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.239251    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 19 11:34:07.239386 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.239260    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 19 11:34:07.240510 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.240485    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-136.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 19 11:34:07.240510 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.240494    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 19 11:34:07.243127 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.243110    2559 server.go:1268] "Started kubelet"
Sep 19 11:34:07.243223 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.243174    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 19 11:34:07.243270 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.243202    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 19 11:34:07.243270 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.243263    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 19 11:34:07.243771 ip-10-0-140-136 systemd[1]: Started Kubernetes Kubelet.
Sep 19 11:34:07.244465 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.244274    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l9ql4"
Sep 19 11:34:07.244833 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.244675    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 19 11:34:07.245915 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.245900    2559 server.go:310] "Adding debug handlers to kubelet server"
Sep 19 11:34:07.246082 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.246067    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-136.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 19 11:34:07.249947 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.249931    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 19 11:34:07.250243 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.249105    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-136.ec2.internal.18d6b5aad4a64043  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-136.ec2.internal,UID:ip-10-0-140-136.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-136.ec2.internal,},FirstTimestamp:2026-09-19 11:34:07.243083843 +0000 UTC m=+0.373402211,LastTimestamp:2026-09-19 11:34:07.243083843 +0000 UTC m=+0.373402211,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-136.ec2.internal,}"
Sep 19 11:34:07.255224 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.255201    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 19 11:34:07.255929 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.255904    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 19 11:34:07.256066 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.256045    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-136.ec2.internal\" not found"
Sep 19 11:34:07.256761 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.256743    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 19 11:34:07.256761 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.256745    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 19 11:34:07.256761 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.256766    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 19 11:34:07.256905 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.256824    2559 reconstruct.go:97] "Volume reconstruction finished"
Sep 19 11:34:07.256905 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.256831    2559 reconciler.go:29] "Reconciler: start to sync state"
Sep 19 11:34:07.258427 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.258410    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 11:34:07.259945 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.259928    2559 factory.go:55] Registering systemd factory
Sep 19 11:34:07.260004 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.259950    2559 factory.go:223] Registration of the systemd container factory successfully
Sep 19 11:34:07.260219 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.260206    2559 factory.go:153] Registering CRI-O factory
Sep 19 11:34:07.260256 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.260222    2559 factory.go:223] Registration of the crio container factory successfully
Sep 19 11:34:07.260284 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.260263    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 19 11:34:07.260284 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.260282    2559 factory.go:103] Registering Raw factory
Sep 19 11:34:07.260349 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.260292    2559 manager.go:1196] Started watching for new ooms in manager
Sep 19 11:34:07.260678 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.260634    2559 manager.go:349] Starting recovery of all containers
Sep 19 11:34:07.261231 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.261139    2559 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-136.ec2.internal\" not found" node="ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.272373 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.272353    2559 manager.go:354] Recovery completed
Sep 19 11:34:07.276760 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.276747    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 11:34:07.279345 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.279329    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 11:34:07.279414 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.279359    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 11:34:07.279414 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.279372    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasSufficientPID"
Sep 19 11:34:07.279841 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.279825    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 19 11:34:07.279841 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.279840    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 19 11:34:07.279953 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.279858    2559 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 11:34:07.279953 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.279872    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 11:34:07.280604 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.280589    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 11:34:07.280662 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.280622    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 11:34:07.280662 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.280632    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasSufficientPID"
Sep 19 11:34:07.282074 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.282060    2559 policy_none.go:49] "None policy: Start"
Sep 19 11:34:07.282135 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.282078    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 19 11:34:07.282135 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.282091    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 19 11:34:07.284111 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.284092    2559 policy_none.go:47] "Start"
Sep 19 11:34:07.332831 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.325778    2559 manager.go:337] "Starting Device Plugin manager"
Sep 19 11:34:07.332831 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.325811    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 19 11:34:07.332831 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.325861    2559 server.go:87] "Starting device plugin registration server"
Sep 19 11:34:07.332831 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.326149    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 19 11:34:07.332831 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.326158    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 19 11:34:07.332831 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.326271    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 19 11:34:07.332831 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.326368    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 19 11:34:07.332831 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.326380    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 19 11:34:07.332831 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.326769    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 19 11:34:07.332831 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.326812    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-136.ec2.internal\" not found"
Sep 19 11:34:07.379038 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.379015    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 19 11:34:07.380337 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.380323    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 19 11:34:07.380391 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.380367    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 19 11:34:07.380391 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.380385    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 19 11:34:07.380466 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.380427    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 19 11:34:07.382667 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.382650    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 11:34:07.426650 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.426585    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 11:34:07.427394 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.427377    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 11:34:07.427471 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.427406    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 11:34:07.427471 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.427424    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasSufficientPID"
Sep 19 11:34:07.427471 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.427448    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.434785 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.434773    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.434843 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.434792    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-136.ec2.internal\": node \"ip-10-0-140-136.ec2.internal\" not found"
Sep 19 11:34:07.450348 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.450327    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-136.ec2.internal\" not found"
Sep 19 11:34:07.480943 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.480919    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-136.ec2.internal"]
Sep 19 11:34:07.480999 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.480979    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 11:34:07.482400 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.482385    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 11:34:07.482456 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.482410    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 11:34:07.482456 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.482424    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasSufficientPID"
Sep 19 11:34:07.484809 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.484797    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 11:34:07.484924 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.484911    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.484967 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.484941    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 11:34:07.485586 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.485573    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 11:34:07.485649 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.485594    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 11:34:07.485649 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.485603    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasSufficientPID"
Sep 19 11:34:07.485649 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.485626    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 11:34:07.485649 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.485647    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 11:34:07.485778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.485658    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasSufficientPID"
Sep 19 11:34:07.487911 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.487896    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.487965 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.487929    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 11:34:07.488649 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.488631    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 11:34:07.488649 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.488652    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 11:34:07.488783 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.488663    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeHasSufficientPID"
Sep 19 11:34:07.510080 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.510065    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-136.ec2.internal\" not found" node="ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.514350 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.514333    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-136.ec2.internal\" not found" node="ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.550714 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.550698    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-136.ec2.internal\" not found"
Sep 19 11:34:07.558389 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.558369    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e3afcac4a342767f8c0136edf032e60-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal\" (UID: \"4e3afcac4a342767f8c0136edf032e60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.558479 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.558396    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e3afcac4a342767f8c0136edf032e60-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal\" (UID: \"4e3afcac4a342767f8c0136edf032e60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.558479 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.558413    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8e524724c540c684f761004a0a6e1643-config\") pod \"kube-apiserver-proxy-ip-10-0-140-136.ec2.internal\" (UID: \"8e524724c540c684f761004a0a6e1643\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.651667 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.651636    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-136.ec2.internal\" not found"
Sep 19 11:34:07.658986 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.658968    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e3afcac4a342767f8c0136edf032e60-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal\" (UID: \"4e3afcac4a342767f8c0136edf032e60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.659058 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.659002    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e3afcac4a342767f8c0136edf032e60-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal\" (UID: \"4e3afcac4a342767f8c0136edf032e60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.659058 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.659008    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e3afcac4a342767f8c0136edf032e60-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal\" (UID: \"4e3afcac4a342767f8c0136edf032e60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.659058 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.659020    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8e524724c540c684f761004a0a6e1643-config\") pod \"kube-apiserver-proxy-ip-10-0-140-136.ec2.internal\" (UID: \"8e524724c540c684f761004a0a6e1643\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.659200 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.659062    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8e524724c540c684f761004a0a6e1643-config\") pod \"kube-apiserver-proxy-ip-10-0-140-136.ec2.internal\" (UID: \"8e524724c540c684f761004a0a6e1643\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.659200 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.659088    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e3afcac4a342767f8c0136edf032e60-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal\" (UID: \"4e3afcac4a342767f8c0136edf032e60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.752416 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.752357    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-136.ec2.internal\" not found"
Sep 19 11:34:07.813896 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.813870    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.818477 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:07.818461    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-136.ec2.internal"
Sep 19 11:34:07.853413 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.853382    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-136.ec2.internal\" not found"
Sep 19 11:34:07.953923 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:07.953899    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-136.ec2.internal\" not found"
Sep 19 11:34:08.054422 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:08.054360    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-136.ec2.internal\" not found"
Sep 19 11:34:08.154942 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:08.154911    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-136.ec2.internal\" not found"
Sep 19 11:34:08.184352 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:08.184330    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 19 11:34:08.184718 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:08.184458    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 11:34:08.184718 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:08.184485    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 11:34:08.250486 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:08.250442    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-18 11:29:08 +0000 UTC" deadline="2028-04-12 11:34:24.901680646 +0000 UTC"
Sep 19 11:34:08.250486 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:08.250482    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13704h0m16.651200765s"
Sep 19 11:34:08.255812 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:08.255794    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-136.ec2.internal\" not found"
Sep 19 11:34:08.291346 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:08.291328    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 11:34:08.356496 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:08.356438    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal"
Sep 19 11:34:08.365653 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:08.365630    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 11:34:08.367809 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:08.367793    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 11:34:08.368674 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:08.368660    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-136.ec2.internal"
Sep 19 11:34:08.376644 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:08.376626    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 11:34:08.431919 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:08.431885    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e3afcac4a342767f8c0136edf032e60.slice/crio-56151264f239c7902fb3437205fefb4ef7cd360688bc75131d909605483e4fb7 WatchSource:0}: Error finding container 56151264f239c7902fb3437205fefb4ef7cd360688bc75131d909605483e4fb7: Status 404 returned error can't find the container with id 56151264f239c7902fb3437205fefb4ef7cd360688bc75131d909605483e4fb7
Sep 19 11:34:08.432434 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:08.432416    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e524724c540c684f761004a0a6e1643.slice/crio-9e59c67cad3a78272670a1c7943e88b36fc6559ebc284589c50bc3dac3e01392 WatchSource:0}: Error finding container 9e59c67cad3a78272670a1c7943e88b36fc6559ebc284589c50bc3dac3e01392: Status 404 returned error can't find the container with id 9e59c67cad3a78272670a1c7943e88b36fc6559ebc284589c50bc3dac3e01392
Sep 19 11:34:08.436925 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:08.436912    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 11:34:10.390219 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.390125    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 11:34:10.441143 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.441112    2559 apiserver.go:52] "Watching apiserver"
Sep 19 11:34:10.448560 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.448534    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 19 11:34:10.450130 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.449716    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-w6hmz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58","openshift-cluster-node-tuning-operator/tuned-zj5bk","openshift-dns/node-resolver-fjmwn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal","openshift-network-operator/iptables-alerter-9tfzm","openshift-ovn-kubernetes/ovnkube-node-mx5t8","kube-system/konnectivity-agent-mxrb8","kube-system/kube-apiserver-proxy-ip-10-0-140-136.ec2.internal","openshift-image-registry/node-ca-2jpjr","openshift-multus/multus-98pk7","openshift-multus/multus-additional-cni-plugins-bd7pv","openshift-multus/network-metrics-daemon-fm8xq","openshift-network-diagnostics/network-check-target-lslq5"]
Sep 19 11:34:10.452422 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.452401    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2jpjr"
Sep 19 11:34:10.454569 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.454536    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.454669 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.454622    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 19 11:34:10.454979 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.454959    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 19 11:34:10.455074 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.455040    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7bcvk\""
Sep 19 11:34:10.455158 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.455140    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 19 11:34:10.456582 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.456565    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 19 11:34:10.456811 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.456772    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4pmfh\""
Sep 19 11:34:10.457095 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.457059    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 19 11:34:10.457305 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.457291    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 19 11:34:10.457541 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.457522    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 19 11:34:10.459836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.459809    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.462045 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.461901    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hnchx\""
Sep 19 11:34:10.462045 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.461928    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:10.462045 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.461940    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 19 11:34:10.462045 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:10.462006    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:10.462297 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.462150    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 19 11:34:10.464225 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.463995    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 19 11:34:10.464225 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.464085    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mxrb8"
Sep 19 11:34:10.466435 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.466420    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 19 11:34:10.466435 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.466431    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 19 11:34:10.466612 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.466601    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zxfrx\""
Sep 19 11:34:10.466958 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.466944    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9tfzm"
Sep 19 11:34:10.469040 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.469022    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 19 11:34:10.469270 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.469255    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-69z9h\""
Sep 19 11:34:10.469441 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.469412    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 19 11:34:10.469512 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.469478    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 19 11:34:10.469610 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.469595    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.473457 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.472331    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9vs4j\""
Sep 19 11:34:10.473457 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.472574    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 19 11:34:10.473457 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.472740    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 19 11:34:10.473457 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.472887    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:10.473457 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:10.472948    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:10.474636 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.474547    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-cnibin\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.474636 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.474581    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-run-multus-certs\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.474636 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.474604    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-run\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.474636 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.474624    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-host\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.474900 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.474646    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7clqj\" (UniqueName: \"kubernetes.io/projected/af32c67c-4404-440d-986f-ec4663e165e6-kube-api-access-7clqj\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.474900 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.474668    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-sysctl-d\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.474900 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.474689    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-var-lib-cni-bin\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.474900 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.474709    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-hostroot\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.474900 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.474730    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-etc-kubernetes\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.474900 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.474750    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-modprobe-d\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.474900 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.474772    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-kubernetes\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.474900 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.474794    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4002479e-43d6-4fbb-9e74-f9a9d0c02c9c-host\") pod \"node-ca-2jpjr\" (UID: \"4002479e-43d6-4fbb-9e74-f9a9d0c02c9c\") " pod="openshift-image-registry/node-ca-2jpjr"
Sep 19 11:34:10.474900 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.474818    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4002479e-43d6-4fbb-9e74-f9a9d0c02c9c-serviceca\") pod \"node-ca-2jpjr\" (UID: \"4002479e-43d6-4fbb-9e74-f9a9d0c02c9c\") " pod="openshift-image-registry/node-ca-2jpjr"
Sep 19 11:34:10.474900 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.474843    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpjxg\" (UniqueName: \"kubernetes.io/projected/4002479e-43d6-4fbb-9e74-f9a9d0c02c9c-kube-api-access-fpjxg\") pod \"node-ca-2jpjr\" (UID: \"4002479e-43d6-4fbb-9e74-f9a9d0c02c9c\") " pod="openshift-image-registry/node-ca-2jpjr"
Sep 19 11:34:10.474900 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.474884    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6976d909-6428-474c-8e7c-0c012b8e23f8-cni-binary-copy\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.475523 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475504    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.475523 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475508    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-run-k8s-cni-cncf-io\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.475630 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475536    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-multus-conf-dir\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.475630 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475559    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af32c67c-4404-440d-986f-ec4663e165e6-tmp\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.475630 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475613    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3c490d87-bfde-4aa7-b297-e71e3f1c8056-konnectivity-ca\") pod \"konnectivity-agent-mxrb8\" (UID: \"3c490d87-bfde-4aa7-b297-e71e3f1c8056\") " pod="kube-system/konnectivity-agent-mxrb8"
Sep 19 11:34:10.475775 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475637    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d3ad7990-91f1-43ee-b69b-e1f4f04f37e7-iptables-alerter-script\") pod \"iptables-alerter-9tfzm\" (UID: \"d3ad7990-91f1-43ee-b69b-e1f4f04f37e7\") " pod="openshift-network-operator/iptables-alerter-9tfzm"
Sep 19 11:34:10.475775 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475660    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-system-cni-dir\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.475775 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475683    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-run-netns\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.475775 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475724    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brnlp\" (UniqueName: \"kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp\") pod \"network-check-target-lslq5\" (UID: \"9ba68c1a-f003-473e-9d35-d86153d8c160\") " pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:10.475775 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475759    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3ad7990-91f1-43ee-b69b-e1f4f04f37e7-host-slash\") pod \"iptables-alerter-9tfzm\" (UID: \"d3ad7990-91f1-43ee-b69b-e1f4f04f37e7\") " pod="openshift-network-operator/iptables-alerter-9tfzm"
Sep 19 11:34:10.476009 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475783    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrrdf\" (UniqueName: \"kubernetes.io/projected/6976d909-6428-474c-8e7c-0c012b8e23f8-kube-api-access-hrrdf\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.476009 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475805    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-sysconfig\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.476009 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475828    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/af32c67c-4404-440d-986f-ec4663e165e6-etc-tuned\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.476009 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475850    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-multus-socket-dir-parent\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.476009 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475878    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-sys\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.476009 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475901    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-lib-modules\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.476009 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475923    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3c490d87-bfde-4aa7-b297-e71e3f1c8056-agent-certs\") pod \"konnectivity-agent-mxrb8\" (UID: \"3c490d87-bfde-4aa7-b297-e71e3f1c8056\") " pod="kube-system/konnectivity-agent-mxrb8"
Sep 19 11:34:10.476009 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475946    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-multus-cni-dir\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.476009 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475971    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-var-lib-cni-multus\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.476009 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.475994    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-var-lib-kubelet\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.476481 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.476019    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6976d909-6428-474c-8e7c-0c012b8e23f8-multus-daemon-config\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.476481 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.476043    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-sysctl-conf\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.476481 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.476066    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-systemd\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.476481 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.476104    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-var-lib-kubelet\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.476481 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.476136    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ww2c\" (UniqueName: \"kubernetes.io/projected/d3ad7990-91f1-43ee-b69b-e1f4f04f37e7-kube-api-access-5ww2c\") pod \"iptables-alerter-9tfzm\" (UID: \"d3ad7990-91f1-43ee-b69b-e1f4f04f37e7\") " pod="openshift-network-operator/iptables-alerter-9tfzm"
Sep 19 11:34:10.477528 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.477372    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 19 11:34:10.477528 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.477494    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 19 11:34:10.477676 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.477631    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:10.477729 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:10.477689    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:10.477862 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.477830    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 19 11:34:10.478244 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.478046    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 11:34:10.478244 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.478058    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5zwv7\""
Sep 19 11:34:10.480089 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.480070    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.481932 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.481915    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 19 11:34:10.482104 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.482084    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 19 11:34:10.482358 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.482342    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fjmwn"
Sep 19 11:34:10.482677 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.482617    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 19 11:34:10.482844 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.482827    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 19 11:34:10.482908 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.482855    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 19 11:34:10.483532 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.483516    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 19 11:34:10.483601 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.483546    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dw6nj\""
Sep 19 11:34:10.484827 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.484631    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 19 11:34:10.484827 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.484669    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 19 11:34:10.484827 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.484680    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pb4jp\""
Sep 19 11:34:10.499011 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.498991    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-knhp9"
Sep 19 11:34:10.508711 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.508686    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-knhp9"
Sep 19 11:34:10.516594 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.516573    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 11:34:10.565815 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.565781    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 19 11:34:10.576790 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.576764    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3ad7990-91f1-43ee-b69b-e1f4f04f37e7-host-slash\") pod \"iptables-alerter-9tfzm\" (UID: \"d3ad7990-91f1-43ee-b69b-e1f4f04f37e7\") " pod="openshift-network-operator/iptables-alerter-9tfzm"
Sep 19 11:34:10.576921 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.576795    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fdf9b2d4-b351-4020-8ee5-4b00b841066d-kubelet-config\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:10.576921 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.576821    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-etc-openvswitch\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.576921 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.576845    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-socket-dir\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.576921 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.576868    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-etc-selinux\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.576921 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.576877    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3ad7990-91f1-43ee-b69b-e1f4f04f37e7-host-slash\") pod \"iptables-alerter-9tfzm\" (UID: \"d3ad7990-91f1-43ee-b69b-e1f4f04f37e7\") " pod="openshift-network-operator/iptables-alerter-9tfzm"
Sep 19 11:34:10.576921 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.576910    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/af32c67c-4404-440d-986f-ec4663e165e6-etc-tuned\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.577230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.576943    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fdf9b2d4-b351-4020-8ee5-4b00b841066d-dbus\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:10.577230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.576971    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c92467c1-64b1-4516-bf34-4d49a767711a-ovnkube-script-lib\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.577230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.576995    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-device-dir\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.577230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577020    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-multus-socket-dir-parent\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.577230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577042    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-sys\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.577230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577064    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20b32973-0255-40ad-a150-fa451be068ec-cni-binary-copy\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.577230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577072    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-multus-socket-dir-parent\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.577230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577087    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:10.577230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577111    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5hbk\" (UniqueName: \"kubernetes.io/projected/c92467c1-64b1-4516-bf34-4d49a767711a-kube-api-access-x5hbk\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.577230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577113    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-sys\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.577230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577148    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-var-lib-kubelet\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.577230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577176    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75cc2\" (UniqueName: \"kubernetes.io/projected/20b32973-0255-40ad-a150-fa451be068ec-kube-api-access-75cc2\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.577230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577181    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-var-lib-kubelet\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.577230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577221    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-cnibin\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.577230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577240    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-sysctl-d\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577255    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-etc-kubernetes\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577270    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-modprobe-d\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577269    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-cnibin\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577287    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs\") pod \"network-metrics-daemon-fm8xq\" (UID: \"50a0f2dc-b541-449e-8e88-8f871f1b30b5\") " pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577306    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-cni-bin\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577323    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af32c67c-4404-440d-986f-ec4663e165e6-tmp\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577336    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-etc-kubernetes\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577344    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-kubelet\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577378    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-modprobe-d\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577406    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577485    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-sysctl-d\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577519    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-system-cni-dir\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577552    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577558    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-run-netns\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577623    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brnlp\" (UniqueName: \"kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp\") pod \"network-check-target-lslq5\" (UID: \"9ba68c1a-f003-473e-9d35-d86153d8c160\") " pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577652    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-system-cni-dir\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577652    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20b32973-0255-40ad-a150-fa451be068ec-system-cni-dir\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.577949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577834    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-run-systemd\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577880    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrrdf\" (UniqueName: \"kubernetes.io/projected/6976d909-6428-474c-8e7c-0c012b8e23f8-kube-api-access-hrrdf\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577935    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-sysconfig\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577955    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-run-netns\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.577963    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-run-ovn-kubernetes\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578002    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ed690d2-ca31-447d-bda0-ee83ae9efc22-tmp-dir\") pod \"node-resolver-fjmwn\" (UID: \"6ed690d2-ca31-447d-bda0-ee83ae9efc22\") " pod="openshift-dns/node-resolver-fjmwn"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578028    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-sys-fs\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578058    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-sysconfig\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578097    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-lib-modules\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578120    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3c490d87-bfde-4aa7-b297-e71e3f1c8056-agent-certs\") pod \"konnectivity-agent-mxrb8\" (UID: \"3c490d87-bfde-4aa7-b297-e71e3f1c8056\") " pod="kube-system/konnectivity-agent-mxrb8"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578144    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-systemd-units\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578167    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-run-netns\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578208    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-run-ovn\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578246    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-cni-netd\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578288    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-lib-modules\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578337    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c92467c1-64b1-4516-bf34-4d49a767711a-ovn-node-metrics-cert\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578362    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6ed690d2-ca31-447d-bda0-ee83ae9efc22-hosts-file\") pod \"node-resolver-fjmwn\" (UID: \"6ed690d2-ca31-447d-bda0-ee83ae9efc22\") " pod="openshift-dns/node-resolver-fjmwn"
Sep 19 11:34:10.578702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578408    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-multus-cni-dir\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578445    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-var-lib-cni-multus\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578474    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-var-lib-cni-multus\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578528    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6976d909-6428-474c-8e7c-0c012b8e23f8-multus-daemon-config\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578572    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-multus-cni-dir\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578581    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-sysctl-conf\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578683    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-sysctl-conf\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578706    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-systemd\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578745    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-var-lib-kubelet\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578779    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ww2c\" (UniqueName: \"kubernetes.io/projected/d3ad7990-91f1-43ee-b69b-e1f4f04f37e7-kube-api-access-5ww2c\") pod \"iptables-alerter-9tfzm\" (UID: \"d3ad7990-91f1-43ee-b69b-e1f4f04f37e7\") " pod="openshift-network-operator/iptables-alerter-9tfzm"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578783    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-systemd\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578804    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-var-lib-kubelet\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578856    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-node-log\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.578896    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-run-multus-certs\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579006    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-run\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579034    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-host\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579042    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-run-multus-certs\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579077    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7clqj\" (UniqueName: \"kubernetes.io/projected/af32c67c-4404-440d-986f-ec4663e165e6-kube-api-access-7clqj\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.579572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579111    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-host\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579115    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20b32973-0255-40ad-a150-fa451be068ec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579121    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6976d909-6428-474c-8e7c-0c012b8e23f8-multus-daemon-config\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579128    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-run\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579154    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-log-socket\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579213    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c92467c1-64b1-4516-bf34-4d49a767711a-ovnkube-config\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579261    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c92467c1-64b1-4516-bf34-4d49a767711a-env-overrides\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579302    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-var-lib-cni-bin\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579345    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-var-lib-cni-bin\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579354    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-hostroot\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579420    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-kubernetes\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579448    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/20b32973-0255-40ad-a150-fa451be068ec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579475    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2cpk\" (UniqueName: \"kubernetes.io/projected/50a0f2dc-b541-449e-8e88-8f871f1b30b5-kube-api-access-r2cpk\") pod \"network-metrics-daemon-fm8xq\" (UID: \"50a0f2dc-b541-449e-8e88-8f871f1b30b5\") " pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579488    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-hostroot\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579505    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-var-lib-openvswitch\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579513    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af32c67c-4404-440d-986f-ec4663e165e6-etc-kubernetes\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579534    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-run-openvswitch\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.580470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579582    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lftv\" (UniqueName: \"kubernetes.io/projected/6ed690d2-ca31-447d-bda0-ee83ae9efc22-kube-api-access-6lftv\") pod \"node-resolver-fjmwn\" (UID: \"6ed690d2-ca31-447d-bda0-ee83ae9efc22\") " pod="openshift-dns/node-resolver-fjmwn"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579788    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4002479e-43d6-4fbb-9e74-f9a9d0c02c9c-host\") pod \"node-ca-2jpjr\" (UID: \"4002479e-43d6-4fbb-9e74-f9a9d0c02c9c\") " pod="openshift-image-registry/node-ca-2jpjr"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579813    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4002479e-43d6-4fbb-9e74-f9a9d0c02c9c-serviceca\") pod \"node-ca-2jpjr\" (UID: \"4002479e-43d6-4fbb-9e74-f9a9d0c02c9c\") " pod="openshift-image-registry/node-ca-2jpjr"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579837    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpjxg\" (UniqueName: \"kubernetes.io/projected/4002479e-43d6-4fbb-9e74-f9a9d0c02c9c-kube-api-access-fpjxg\") pod \"node-ca-2jpjr\" (UID: \"4002479e-43d6-4fbb-9e74-f9a9d0c02c9c\") " pod="openshift-image-registry/node-ca-2jpjr"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579859    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6976d909-6428-474c-8e7c-0c012b8e23f8-cni-binary-copy\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579883    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-run-k8s-cni-cncf-io\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579907    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-multus-conf-dir\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579932    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20b32973-0255-40ad-a150-fa451be068ec-cnibin\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579957    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579982    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3c490d87-bfde-4aa7-b297-e71e3f1c8056-konnectivity-ca\") pod \"konnectivity-agent-mxrb8\" (UID: \"3c490d87-bfde-4aa7-b297-e71e3f1c8056\") " pod="kube-system/konnectivity-agent-mxrb8"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.580005    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d3ad7990-91f1-43ee-b69b-e1f4f04f37e7-iptables-alerter-script\") pod \"iptables-alerter-9tfzm\" (UID: \"d3ad7990-91f1-43ee-b69b-e1f4f04f37e7\") " pod="openshift-network-operator/iptables-alerter-9tfzm"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.580031    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/20b32973-0255-40ad-a150-fa451be068ec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.579907    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4002479e-43d6-4fbb-9e74-f9a9d0c02c9c-host\") pod \"node-ca-2jpjr\" (UID: \"4002479e-43d6-4fbb-9e74-f9a9d0c02c9c\") " pod="openshift-image-registry/node-ca-2jpjr"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.580061    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-slash\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.580083    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-registration-dir\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.580106    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxf48\" (UniqueName: \"kubernetes.io/projected/99dfbfd8-d697-427c-bc84-0932af7c82ad-kube-api-access-gxf48\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.580265    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-host-run-k8s-cni-cncf-io\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.581361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.580352    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4002479e-43d6-4fbb-9e74-f9a9d0c02c9c-serviceca\") pod \"node-ca-2jpjr\" (UID: \"4002479e-43d6-4fbb-9e74-f9a9d0c02c9c\") " pod="openshift-image-registry/node-ca-2jpjr"
Sep 19 11:34:10.582230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.580569    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6976d909-6428-474c-8e7c-0c012b8e23f8-multus-conf-dir\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.582230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.580715    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3c490d87-bfde-4aa7-b297-e71e3f1c8056-konnectivity-ca\") pod \"konnectivity-agent-mxrb8\" (UID: \"3c490d87-bfde-4aa7-b297-e71e3f1c8056\") " pod="kube-system/konnectivity-agent-mxrb8"
Sep 19 11:34:10.582230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.580863    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6976d909-6428-474c-8e7c-0c012b8e23f8-cni-binary-copy\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.582230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.581544    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d3ad7990-91f1-43ee-b69b-e1f4f04f37e7-iptables-alerter-script\") pod \"iptables-alerter-9tfzm\" (UID: \"d3ad7990-91f1-43ee-b69b-e1f4f04f37e7\") " pod="openshift-network-operator/iptables-alerter-9tfzm"
Sep 19 11:34:10.582230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.581925    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af32c67c-4404-440d-986f-ec4663e165e6-tmp\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.582230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.581935    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/af32c67c-4404-440d-986f-ec4663e165e6-etc-tuned\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.582418 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.582303    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3c490d87-bfde-4aa7-b297-e71e3f1c8056-agent-certs\") pod \"konnectivity-agent-mxrb8\" (UID: \"3c490d87-bfde-4aa7-b297-e71e3f1c8056\") " pod="kube-system/konnectivity-agent-mxrb8"
Sep 19 11:34:10.584578 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:10.584539    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 11:34:10.584578 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:10.584571    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 11:34:10.584578 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:10.584582    2559 projected.go:196] Error preparing data for projected volume kube-api-access-brnlp for pod openshift-network-diagnostics/network-check-target-lslq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 11:34:10.584864 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:10.584626    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp podName:9ba68c1a-f003-473e-9d35-d86153d8c160 nodeName:}" failed. No retries permitted until 2026-09-19 11:34:11.084612508 +0000 UTC m=+3.006925367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-brnlp" (UniqueName: "kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp") pod "network-check-target-lslq5" (UID: "9ba68c1a-f003-473e-9d35-d86153d8c160") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 11:34:10.587421 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.587398    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7clqj\" (UniqueName: \"kubernetes.io/projected/af32c67c-4404-440d-986f-ec4663e165e6-kube-api-access-7clqj\") pod \"tuned-zj5bk\" (UID: \"af32c67c-4404-440d-986f-ec4663e165e6\") " pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.587588 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.587571    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ww2c\" (UniqueName: \"kubernetes.io/projected/d3ad7990-91f1-43ee-b69b-e1f4f04f37e7-kube-api-access-5ww2c\") pod \"iptables-alerter-9tfzm\" (UID: \"d3ad7990-91f1-43ee-b69b-e1f4f04f37e7\") " pod="openshift-network-operator/iptables-alerter-9tfzm"
Sep 19 11:34:10.587762 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.587743    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrrdf\" (UniqueName: \"kubernetes.io/projected/6976d909-6428-474c-8e7c-0c012b8e23f8-kube-api-access-hrrdf\") pod \"multus-98pk7\" (UID: \"6976d909-6428-474c-8e7c-0c012b8e23f8\") " pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.588000 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.587980    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpjxg\" (UniqueName: \"kubernetes.io/projected/4002479e-43d6-4fbb-9e74-f9a9d0c02c9c-kube-api-access-fpjxg\") pod \"node-ca-2jpjr\" (UID: \"4002479e-43d6-4fbb-9e74-f9a9d0c02c9c\") " pod="openshift-image-registry/node-ca-2jpjr"
Sep 19 11:34:10.593341 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.593294    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-136.ec2.internal" event={"ID":"8e524724c540c684f761004a0a6e1643","Type":"ContainerStarted","Data":"9e59c67cad3a78272670a1c7943e88b36fc6559ebc284589c50bc3dac3e01392"}
Sep 19 11:34:10.594598 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.594573    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal" event={"ID":"4e3afcac4a342767f8c0136edf032e60","Type":"ContainerStarted","Data":"56151264f239c7902fb3437205fefb4ef7cd360688bc75131d909605483e4fb7"}
Sep 19 11:34:10.680653 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680560    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-node-log\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.680653 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680605    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20b32973-0255-40ad-a150-fa451be068ec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.680653 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680622    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-log-socket\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.680914 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680659    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c92467c1-64b1-4516-bf34-4d49a767711a-ovnkube-config\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.680914 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680682    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c92467c1-64b1-4516-bf34-4d49a767711a-env-overrides\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.680914 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680687    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-node-log\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.680914 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680699    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/20b32973-0255-40ad-a150-fa451be068ec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.680914 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680743    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2cpk\" (UniqueName: \"kubernetes.io/projected/50a0f2dc-b541-449e-8e88-8f871f1b30b5-kube-api-access-r2cpk\") pod \"network-metrics-daemon-fm8xq\" (UID: \"50a0f2dc-b541-449e-8e88-8f871f1b30b5\") " pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:10.680914 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680769    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-var-lib-openvswitch\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.680914 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680793    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-run-openvswitch\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.680914 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680814    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lftv\" (UniqueName: \"kubernetes.io/projected/6ed690d2-ca31-447d-bda0-ee83ae9efc22-kube-api-access-6lftv\") pod \"node-resolver-fjmwn\" (UID: \"6ed690d2-ca31-447d-bda0-ee83ae9efc22\") " pod="openshift-dns/node-resolver-fjmwn"
Sep 19 11:34:10.680914 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680847    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20b32973-0255-40ad-a150-fa451be068ec-cnibin\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.680914 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680871    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.680914 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680888    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/20b32973-0255-40ad-a150-fa451be068ec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.680914 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680904    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-slash\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.680914 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680919    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-registration-dir\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680946    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxf48\" (UniqueName: \"kubernetes.io/projected/99dfbfd8-d697-427c-bc84-0932af7c82ad-kube-api-access-gxf48\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680970    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fdf9b2d4-b351-4020-8ee5-4b00b841066d-kubelet-config\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.680991    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-etc-openvswitch\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681007    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-socket-dir\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681022    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-etc-selinux\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681042    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fdf9b2d4-b351-4020-8ee5-4b00b841066d-dbus\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681058    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c92467c1-64b1-4516-bf34-4d49a767711a-ovnkube-script-lib\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681072    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-device-dir\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681088    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20b32973-0255-40ad-a150-fa451be068ec-cni-binary-copy\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681103    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681123    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5hbk\" (UniqueName: \"kubernetes.io/projected/c92467c1-64b1-4516-bf34-4d49a767711a-kube-api-access-x5hbk\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681146    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75cc2\" (UniqueName: \"kubernetes.io/projected/20b32973-0255-40ad-a150-fa451be068ec-kube-api-access-75cc2\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681166    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/20b32973-0255-40ad-a150-fa451be068ec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681177    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs\") pod \"network-metrics-daemon-fm8xq\" (UID: \"50a0f2dc-b541-449e-8e88-8f871f1b30b5\") " pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681213    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-cni-bin\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681244    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-cni-bin\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.681572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681245    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-kubelet\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681267    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681274    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-kubelet\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681304    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20b32973-0255-40ad-a150-fa451be068ec-system-cni-dir\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681326    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-run-systemd\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681354    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-run-ovn-kubernetes\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681379    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ed690d2-ca31-447d-bda0-ee83ae9efc22-tmp-dir\") pod \"node-resolver-fjmwn\" (UID: \"6ed690d2-ca31-447d-bda0-ee83ae9efc22\") " pod="openshift-dns/node-resolver-fjmwn"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681402    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-sys-fs\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681438    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-systemd-units\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681457    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-run-netns\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681472    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-run-ovn\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681491    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-cni-netd\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681505    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c92467c1-64b1-4516-bf34-4d49a767711a-ovn-node-metrics-cert\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681521    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6ed690d2-ca31-447d-bda0-ee83ae9efc22-hosts-file\") pod \"node-resolver-fjmwn\" (UID: \"6ed690d2-ca31-447d-bda0-ee83ae9efc22\") " pod="openshift-dns/node-resolver-fjmwn"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681605    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6ed690d2-ca31-447d-bda0-ee83ae9efc22-hosts-file\") pod \"node-resolver-fjmwn\" (UID: \"6ed690d2-ca31-447d-bda0-ee83ae9efc22\") " pod="openshift-dns/node-resolver-fjmwn"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681687    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c92467c1-64b1-4516-bf34-4d49a767711a-ovnkube-config\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681749    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20b32973-0255-40ad-a150-fa451be068ec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.682384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681967    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c92467c1-64b1-4516-bf34-4d49a767711a-env-overrides\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.681976    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-var-lib-openvswitch\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682022    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-run-openvswitch\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:10.682070    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:10.682116    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret podName:fdf9b2d4-b351-4020-8ee5-4b00b841066d nodeName:}" failed. No retries permitted until 2026-09-19 11:34:11.182104277 +0000 UTC m=+3.104417135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret") pod "global-pull-secret-syncer-w6hmz" (UID: "fdf9b2d4-b351-4020-8ee5-4b00b841066d") : object "kube-system"/"original-pull-secret" not registered
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682226    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20b32973-0255-40ad-a150-fa451be068ec-cnibin\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682267    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682314    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-registration-dir\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682440    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fdf9b2d4-b351-4020-8ee5-4b00b841066d-kubelet-config\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682485    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-etc-openvswitch\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682571    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-socket-dir\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682625    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20b32973-0255-40ad-a150-fa451be068ec-system-cni-dir\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682664    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-etc-selinux\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682682    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-run-systemd\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682703    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-log-socket\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682713    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-run-ovn-kubernetes\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682735    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.683234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682818    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fdf9b2d4-b351-4020-8ee5-4b00b841066d-dbus\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:10.684030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682917    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ed690d2-ca31-447d-bda0-ee83ae9efc22-tmp-dir\") pod \"node-resolver-fjmwn\" (UID: \"6ed690d2-ca31-447d-bda0-ee83ae9efc22\") " pod="openshift-dns/node-resolver-fjmwn"
Sep 19 11:34:10.684030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.682965    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-sys-fs\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.684030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.683007    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-systemd-units\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.684030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.683040    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-run-netns\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.684030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.683078    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-run-ovn\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.684030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.683113    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-cni-netd\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.684030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.683218    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c92467c1-64b1-4516-bf34-4d49a767711a-ovnkube-script-lib\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.684030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.683277    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/99dfbfd8-d697-427c-bc84-0932af7c82ad-device-dir\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.684030 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:10.683331    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 11:34:10.684030 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:10.683396    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs podName:50a0f2dc-b541-449e-8e88-8f871f1b30b5 nodeName:}" failed. No retries permitted until 2026-09-19 11:34:11.183385254 +0000 UTC m=+3.105698114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs") pod "network-metrics-daemon-fm8xq" (UID: "50a0f2dc-b541-449e-8e88-8f871f1b30b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 11:34:10.684030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.683475    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c92467c1-64b1-4516-bf34-4d49a767711a-host-slash\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.684030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.683760    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20b32973-0255-40ad-a150-fa451be068ec-cni-binary-copy\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.684030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.683806    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/20b32973-0255-40ad-a150-fa451be068ec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.685776 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.685755    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c92467c1-64b1-4516-bf34-4d49a767711a-ovn-node-metrics-cert\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.690250 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.690215    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lftv\" (UniqueName: \"kubernetes.io/projected/6ed690d2-ca31-447d-bda0-ee83ae9efc22-kube-api-access-6lftv\") pod \"node-resolver-fjmwn\" (UID: \"6ed690d2-ca31-447d-bda0-ee83ae9efc22\") " pod="openshift-dns/node-resolver-fjmwn"
Sep 19 11:34:10.691097 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.691068    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75cc2\" (UniqueName: \"kubernetes.io/projected/20b32973-0255-40ad-a150-fa451be068ec-kube-api-access-75cc2\") pod \"multus-additional-cni-plugins-bd7pv\" (UID: \"20b32973-0255-40ad-a150-fa451be068ec\") " pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.691597 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.691578    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2cpk\" (UniqueName: \"kubernetes.io/projected/50a0f2dc-b541-449e-8e88-8f871f1b30b5-kube-api-access-r2cpk\") pod \"network-metrics-daemon-fm8xq\" (UID: \"50a0f2dc-b541-449e-8e88-8f871f1b30b5\") " pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:10.691987 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.691969    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxf48\" (UniqueName: \"kubernetes.io/projected/99dfbfd8-d697-427c-bc84-0932af7c82ad-kube-api-access-gxf48\") pod \"aws-ebs-csi-driver-node-2dj58\" (UID: \"99dfbfd8-d697-427c-bc84-0932af7c82ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.692066 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.692034    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5hbk\" (UniqueName: \"kubernetes.io/projected/c92467c1-64b1-4516-bf34-4d49a767711a-kube-api-access-x5hbk\") pod \"ovnkube-node-mx5t8\" (UID: \"c92467c1-64b1-4516-bf34-4d49a767711a\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.769084 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.769056    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2jpjr"
Sep 19 11:34:10.779742 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.779717    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-98pk7"
Sep 19 11:34:10.789491 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.789469    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zj5bk"
Sep 19 11:34:10.794121 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.794095    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mxrb8"
Sep 19 11:34:10.804794 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.804773    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9tfzm"
Sep 19 11:34:10.812318 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.812300    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bd7pv"
Sep 19 11:34:10.820138 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.820118    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58"
Sep 19 11:34:10.828772 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.828751    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:10.835261 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:10.835243    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fjmwn"
Sep 19 11:34:11.185016 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.184985    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:11.185142 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.185026    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs\") pod \"network-metrics-daemon-fm8xq\" (UID: \"50a0f2dc-b541-449e-8e88-8f871f1b30b5\") " pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:11.185142 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.185049    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brnlp\" (UniqueName: \"kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp\") pod \"network-check-target-lslq5\" (UID: \"9ba68c1a-f003-473e-9d35-d86153d8c160\") " pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:11.185142 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:11.185115    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 11:34:11.185283 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:11.185153    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 11:34:11.185283 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:11.185170    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 11:34:11.185283 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:11.185186    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 11:34:11.185283 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:11.185208    2559 projected.go:196] Error preparing data for projected volume kube-api-access-brnlp for pod openshift-network-diagnostics/network-check-target-lslq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 11:34:11.185283 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:11.185215    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret podName:fdf9b2d4-b351-4020-8ee5-4b00b841066d nodeName:}" failed. No retries permitted until 2026-09-19 11:34:12.185198408 +0000 UTC m=+4.107511280 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret") pod "global-pull-secret-syncer-w6hmz" (UID: "fdf9b2d4-b351-4020-8ee5-4b00b841066d") : object "kube-system"/"original-pull-secret" not registered
Sep 19 11:34:11.185283 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:11.185231    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs podName:50a0f2dc-b541-449e-8e88-8f871f1b30b5 nodeName:}" failed. No retries permitted until 2026-09-19 11:34:12.185224732 +0000 UTC m=+4.107537591 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs") pod "network-metrics-daemon-fm8xq" (UID: "50a0f2dc-b541-449e-8e88-8f871f1b30b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 11:34:11.185283 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:11.185255    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp podName:9ba68c1a-f003-473e-9d35-d86153d8c160 nodeName:}" failed. No retries permitted until 2026-09-19 11:34:12.185241986 +0000 UTC m=+4.107554848 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-brnlp" (UniqueName: "kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp") pod "network-check-target-lslq5" (UID: "9ba68c1a-f003-473e-9d35-d86153d8c160") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 11:34:11.498539 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:11.498509    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf32c67c_4404_440d_986f_ec4663e165e6.slice/crio-13c55e954f560da11f1a42d516c81d4a9c3f74989e38e80458d4d961be1d719f WatchSource:0}: Error finding container 13c55e954f560da11f1a42d516c81d4a9c3f74989e38e80458d4d961be1d719f: Status 404 returned error can't find the container with id 13c55e954f560da11f1a42d516c81d4a9c3f74989e38e80458d4d961be1d719f
Sep 19 11:34:11.504893 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:11.504868    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6976d909_6428_474c_8e7c_0c012b8e23f8.slice/crio-19015545538a99688f7bea4314afc4f4e269036e755129254aa23e355dcec961 WatchSource:0}: Error finding container 19015545538a99688f7bea4314afc4f4e269036e755129254aa23e355dcec961: Status 404 returned error can't find the container with id 19015545538a99688f7bea4314afc4f4e269036e755129254aa23e355dcec961
Sep 19 11:34:11.505504 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:11.505481    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc92467c1_64b1_4516_bf34_4d49a767711a.slice/crio-72ac488f0ee0866a513adff2b2847dc4ed621973c7dbed37b93a695d26fab389 WatchSource:0}: Error finding container 72ac488f0ee0866a513adff2b2847dc4ed621973c7dbed37b93a695d26fab389: Status 404 returned error can't find the container with id 72ac488f0ee0866a513adff2b2847dc4ed621973c7dbed37b93a695d26fab389
Sep 19 11:34:11.506178 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:11.506157    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c490d87_bfde_4aa7_b297_e71e3f1c8056.slice/crio-6ca60d1988b4db81fc8b15651fe584d6a4376345de4af55e8cc1fc313264783a WatchSource:0}: Error finding container 6ca60d1988b4db81fc8b15651fe584d6a4376345de4af55e8cc1fc313264783a: Status 404 returned error can't find the container with id 6ca60d1988b4db81fc8b15651fe584d6a4376345de4af55e8cc1fc313264783a
Sep 19 11:34:11.507285 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:11.507217    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99dfbfd8_d697_427c_bc84_0932af7c82ad.slice/crio-66c77337853a724a2681a29bf036bc8fd673781185036f7e9fde627aa02bbcce WatchSource:0}: Error finding container 66c77337853a724a2681a29bf036bc8fd673781185036f7e9fde627aa02bbcce: Status 404 returned error can't find the container with id 66c77337853a724a2681a29bf036bc8fd673781185036f7e9fde627aa02bbcce
Sep 19 11:34:11.512608 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.511641    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 11:29:10 +0000 UTC" deadline="2028-04-04 07:03:18.404880309 +0000 UTC"
Sep 19 11:34:11.512608 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.511671    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13507h29m6.893213889s"
Sep 19 11:34:11.604080 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.604049    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bd7pv" event={"ID":"20b32973-0255-40ad-a150-fa451be068ec","Type":"ContainerStarted","Data":"352e98c3a3f8cf7d521fa8d7fd8222dc2c54cb3393639ebcb8d3c37b61d19907"}
Sep 19 11:34:11.609606 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.609578    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2jpjr" event={"ID":"4002479e-43d6-4fbb-9e74-f9a9d0c02c9c","Type":"ContainerStarted","Data":"0da4e3053bcf34a66644e8b1626af4dadbe3d2ba90ced40dd8cd4e19391d6b99"}
Sep 19 11:34:11.610720 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.610690    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mxrb8" event={"ID":"3c490d87-bfde-4aa7-b297-e71e3f1c8056","Type":"ContainerStarted","Data":"6ca60d1988b4db81fc8b15651fe584d6a4376345de4af55e8cc1fc313264783a"}
Sep 19 11:34:11.614113 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.614094    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fjmwn" event={"ID":"6ed690d2-ca31-447d-bda0-ee83ae9efc22","Type":"ContainerStarted","Data":"150e8478daee713f51676cfb0f3fc61a723a9f1eab3db3f7a1acb6a647d45a7b"}
Sep 19 11:34:11.615237 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.615219    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9tfzm" event={"ID":"d3ad7990-91f1-43ee-b69b-e1f4f04f37e7","Type":"ContainerStarted","Data":"41bc6661a3588b13e75810feea282c497a184ca45c77bdb70f52898255311b76"}
Sep 19 11:34:11.616735 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.616718    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8" event={"ID":"c92467c1-64b1-4516-bf34-4d49a767711a","Type":"ContainerStarted","Data":"72ac488f0ee0866a513adff2b2847dc4ed621973c7dbed37b93a695d26fab389"}
Sep 19 11:34:11.618287 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.618269    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58" event={"ID":"99dfbfd8-d697-427c-bc84-0932af7c82ad","Type":"ContainerStarted","Data":"66c77337853a724a2681a29bf036bc8fd673781185036f7e9fde627aa02bbcce"}
Sep 19 11:34:11.623465 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.623436    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-98pk7" event={"ID":"6976d909-6428-474c-8e7c-0c012b8e23f8","Type":"ContainerStarted","Data":"19015545538a99688f7bea4314afc4f4e269036e755129254aa23e355dcec961"}
Sep 19 11:34:11.626461 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.626441    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zj5bk" event={"ID":"af32c67c-4404-440d-986f-ec4663e165e6","Type":"ContainerStarted","Data":"13c55e954f560da11f1a42d516c81d4a9c3f74989e38e80458d4d961be1d719f"}
Sep 19 11:34:11.628157 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.628137    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-136.ec2.internal" event={"ID":"8e524724c540c684f761004a0a6e1643","Type":"ContainerStarted","Data":"5033233dfc422536cdc3cb5da91092925c7f9382a7190b3e828b12a3744e46bf"}
Sep 19 11:34:11.643735 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:11.643686    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-136.ec2.internal" podStartSLOduration=2.6436711429999997 podStartE2EDuration="2.643671143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 11:34:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:34:11.643436424 +0000 UTC m=+3.565749307" watchObservedRunningTime="2026-09-19 11:34:11.643671143 +0000 UTC m=+3.565984025"
Sep 19 11:34:12.192422 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:12.192382    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brnlp\" (UniqueName: \"kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp\") pod \"network-check-target-lslq5\" (UID: \"9ba68c1a-f003-473e-9d35-d86153d8c160\") " pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:12.192583 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:12.192484    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:12.192583 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:12.192517    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs\") pod \"network-metrics-daemon-fm8xq\" (UID: \"50a0f2dc-b541-449e-8e88-8f871f1b30b5\") " pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:12.192707 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:12.192659    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 11:34:12.192812 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:12.192723    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs podName:50a0f2dc-b541-449e-8e88-8f871f1b30b5 nodeName:}" failed. No retries permitted until 2026-09-19 11:34:14.192704503 +0000 UTC m=+6.115017387 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs") pod "network-metrics-daemon-fm8xq" (UID: "50a0f2dc-b541-449e-8e88-8f871f1b30b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 11:34:12.193133 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:12.193113    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 11:34:12.193230 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:12.193139    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 11:34:12.193230 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:12.193151    2559 projected.go:196] Error preparing data for projected volume kube-api-access-brnlp for pod openshift-network-diagnostics/network-check-target-lslq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 11:34:12.193230 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:12.193213    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp podName:9ba68c1a-f003-473e-9d35-d86153d8c160 nodeName:}" failed. No retries permitted until 2026-09-19 11:34:14.193181404 +0000 UTC m=+6.115494270 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-brnlp" (UniqueName: "kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp") pod "network-check-target-lslq5" (UID: "9ba68c1a-f003-473e-9d35-d86153d8c160") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 11:34:12.193404 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:12.193281    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 11:34:12.193404 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:12.193314    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret podName:fdf9b2d4-b351-4020-8ee5-4b00b841066d nodeName:}" failed. No retries permitted until 2026-09-19 11:34:14.193304455 +0000 UTC m=+6.115617320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret") pod "global-pull-secret-syncer-w6hmz" (UID: "fdf9b2d4-b351-4020-8ee5-4b00b841066d") : object "kube-system"/"original-pull-secret" not registered
Sep 19 11:34:12.512666 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:12.512558    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 11:29:10 +0000 UTC" deadline="2028-03-14 02:26:54.788722686 +0000 UTC"
Sep 19 11:34:12.512666 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:12.512621    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12998h52m42.276106152s"
Sep 19 11:34:12.591630 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:12.591601    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:12.591778 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:12.591723    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:12.592134 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:12.592116    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:12.592267 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:12.592246    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:12.592341 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:12.592318    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:12.592406 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:12.592390    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:12.646403 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:12.646372    2559 generic.go:356] "Generic (PLEG): container finished" podID="4e3afcac4a342767f8c0136edf032e60" containerID="629ec3f33f1698cab4504019a62ff93b5f8ce6415c61602bfa9fb6b3d414b12e" exitCode=0
Sep 19 11:34:12.646817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:12.646797    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal" event={"ID":"4e3afcac4a342767f8c0136edf032e60","Type":"ContainerDied","Data":"629ec3f33f1698cab4504019a62ff93b5f8ce6415c61602bfa9fb6b3d414b12e"}
Sep 19 11:34:13.674394 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:13.674358    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal" event={"ID":"4e3afcac4a342767f8c0136edf032e60","Type":"ContainerStarted","Data":"87ab36726ad102d74ae44e12732836f1d6ab9a18712b441cc46075ab9ef1ce07"}
Sep 19 11:34:14.211228 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:14.210315    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:14.211228 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:14.210368    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs\") pod \"network-metrics-daemon-fm8xq\" (UID: \"50a0f2dc-b541-449e-8e88-8f871f1b30b5\") " pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:14.211228 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:14.210401    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brnlp\" (UniqueName: \"kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp\") pod \"network-check-target-lslq5\" (UID: \"9ba68c1a-f003-473e-9d35-d86153d8c160\") " pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:14.211228 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:14.210532    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 11:34:14.211228 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:14.210550    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 11:34:14.211228 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:14.210562    2559 projected.go:196] Error preparing data for projected volume kube-api-access-brnlp for pod openshift-network-diagnostics/network-check-target-lslq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 11:34:14.211228 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:14.210622    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp podName:9ba68c1a-f003-473e-9d35-d86153d8c160 nodeName:}" failed. No retries permitted until 2026-09-19 11:34:18.210603441 +0000 UTC m=+10.132916320 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-brnlp" (UniqueName: "kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp") pod "network-check-target-lslq5" (UID: "9ba68c1a-f003-473e-9d35-d86153d8c160") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 11:34:14.211228 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:14.211008    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 11:34:14.211228 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:14.211060    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret podName:fdf9b2d4-b351-4020-8ee5-4b00b841066d nodeName:}" failed. No retries permitted until 2026-09-19 11:34:18.211042062 +0000 UTC m=+10.133354935 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret") pod "global-pull-secret-syncer-w6hmz" (UID: "fdf9b2d4-b351-4020-8ee5-4b00b841066d") : object "kube-system"/"original-pull-secret" not registered
Sep 19 11:34:14.211228 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:14.211115    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 11:34:14.211228 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:14.211145    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs podName:50a0f2dc-b541-449e-8e88-8f871f1b30b5 nodeName:}" failed. No retries permitted until 2026-09-19 11:34:18.211133901 +0000 UTC m=+10.133446775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs") pod "network-metrics-daemon-fm8xq" (UID: "50a0f2dc-b541-449e-8e88-8f871f1b30b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 11:34:14.589110 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:14.589024    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:14.589110 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:14.589053    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:14.589339 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:14.589152    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:14.589339 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:14.589184    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:14.589339 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:14.589293    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:14.589440 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:14.589381    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:16.590374 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:16.590339    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:16.590755 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:16.590461    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:16.590884 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:16.590862    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:16.591017 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:16.590973    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:16.591606 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:16.591586    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:16.591781 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:16.591711    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:18.249042 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:18.248831    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:18.249541 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:18.249062    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs\") pod \"network-metrics-daemon-fm8xq\" (UID: \"50a0f2dc-b541-449e-8e88-8f871f1b30b5\") " pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:18.249541 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:18.249092    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brnlp\" (UniqueName: \"kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp\") pod \"network-check-target-lslq5\" (UID: \"9ba68c1a-f003-473e-9d35-d86153d8c160\") " pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:18.249541 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:18.248995    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 11:34:18.249541 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:18.249166    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret podName:fdf9b2d4-b351-4020-8ee5-4b00b841066d nodeName:}" failed. No retries permitted until 2026-09-19 11:34:26.249146379 +0000 UTC m=+18.171459240 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret") pod "global-pull-secret-syncer-w6hmz" (UID: "fdf9b2d4-b351-4020-8ee5-4b00b841066d") : object "kube-system"/"original-pull-secret" not registered
Sep 19 11:34:18.249541 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:18.249278    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 11:34:18.249541 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:18.249299    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 11:34:18.249541 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:18.249310    2559 projected.go:196] Error preparing data for projected volume kube-api-access-brnlp for pod openshift-network-diagnostics/network-check-target-lslq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 11:34:18.249541 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:18.249357    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp podName:9ba68c1a-f003-473e-9d35-d86153d8c160 nodeName:}" failed. No retries permitted until 2026-09-19 11:34:26.249342508 +0000 UTC m=+18.171655392 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-brnlp" (UniqueName: "kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp") pod "network-check-target-lslq5" (UID: "9ba68c1a-f003-473e-9d35-d86153d8c160") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 11:34:18.249541 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:18.249430    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 11:34:18.249541 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:18.249463    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs podName:50a0f2dc-b541-449e-8e88-8f871f1b30b5 nodeName:}" failed. No retries permitted until 2026-09-19 11:34:26.249451361 +0000 UTC m=+18.171764237 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs") pod "network-metrics-daemon-fm8xq" (UID: "50a0f2dc-b541-449e-8e88-8f871f1b30b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 11:34:18.591826 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:18.591099    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:18.591826 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:18.591222    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:18.591826 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:18.591628    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:18.591826 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:18.591731    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:18.592135 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:18.591878    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:18.592135 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:18.591977    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:20.589184 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:20.589157    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:20.589667 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:20.589208    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:20.589667 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:20.589304    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:20.589667 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:20.589346    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:20.589667 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:20.589424    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:20.589667 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:20.589487    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:22.589150 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:22.588664    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:22.589150 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:22.588705    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:22.589150 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:22.588664    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:22.589150 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:22.588799    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:22.589150 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:22.588890    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:22.589150 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:22.588940    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:24.589532 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:24.589009    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:24.589532 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:24.589042    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:24.589532 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:24.589009    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:24.589532 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:24.589141    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:24.589532 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:24.589249    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:24.589532 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:24.589361    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:26.308841 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:26.308799    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:26.309351 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:26.308856    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs\") pod \"network-metrics-daemon-fm8xq\" (UID: \"50a0f2dc-b541-449e-8e88-8f871f1b30b5\") " pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:26.309351 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:26.308888    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brnlp\" (UniqueName: \"kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp\") pod \"network-check-target-lslq5\" (UID: \"9ba68c1a-f003-473e-9d35-d86153d8c160\") " pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:26.309351 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:26.308968    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 11:34:26.309351 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:26.309021    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 11:34:26.309351 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:26.309033    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 11:34:26.309351 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:26.309050    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 11:34:26.309351 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:26.309064    2559 projected.go:196] Error preparing data for projected volume kube-api-access-brnlp for pod openshift-network-diagnostics/network-check-target-lslq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 11:34:26.309351 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:26.309036    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret podName:fdf9b2d4-b351-4020-8ee5-4b00b841066d nodeName:}" failed. No retries permitted until 2026-09-19 11:34:42.309019996 +0000 UTC m=+34.231332855 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret") pod "global-pull-secret-syncer-w6hmz" (UID: "fdf9b2d4-b351-4020-8ee5-4b00b841066d") : object "kube-system"/"original-pull-secret" not registered
Sep 19 11:34:26.309351 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:26.309123    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs podName:50a0f2dc-b541-449e-8e88-8f871f1b30b5 nodeName:}" failed. No retries permitted until 2026-09-19 11:34:42.309106643 +0000 UTC m=+34.231419505 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs") pod "network-metrics-daemon-fm8xq" (UID: "50a0f2dc-b541-449e-8e88-8f871f1b30b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 11:34:26.309351 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:26.309138    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp podName:9ba68c1a-f003-473e-9d35-d86153d8c160 nodeName:}" failed. No retries permitted until 2026-09-19 11:34:42.309128404 +0000 UTC m=+34.231441265 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-brnlp" (UniqueName: "kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp") pod "network-check-target-lslq5" (UID: "9ba68c1a-f003-473e-9d35-d86153d8c160") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 11:34:26.589417 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:26.589324    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:26.589566 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:26.589324    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:26.589566 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:26.589453    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:26.589566 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:26.589553    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:26.589689 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:26.589324    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:26.589689 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:26.589630    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:28.590145 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:28.590115    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:28.590614 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:28.590257    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:28.590614 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:28.590348    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:28.590614 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:28.590452    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:28.590614 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:28.590497    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:28.590614 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:28.590563    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:29.707378 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.707134    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fjmwn" event={"ID":"6ed690d2-ca31-447d-bda0-ee83ae9efc22","Type":"ContainerStarted","Data":"0fd70a8ee84ebc5f7c5d3570e11f15caf445dc4392eecd002d45176bc3878242"}
Sep 19 11:34:29.711215 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.711141    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8" event={"ID":"c92467c1-64b1-4516-bf34-4d49a767711a","Type":"ContainerStarted","Data":"d9668b721d1afcc2352cc31eb50f6dc4c49407c9418509b289b7730f48176bd4"}
Sep 19 11:34:29.711321 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.711245    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8" event={"ID":"c92467c1-64b1-4516-bf34-4d49a767711a","Type":"ContainerStarted","Data":"904deb36b5db32aa48373880e1bd85860395f8f166add37c44ac2ef8c2e34027"}
Sep 19 11:34:29.711321 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.711288    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8" event={"ID":"c92467c1-64b1-4516-bf34-4d49a767711a","Type":"ContainerStarted","Data":"78cee6c4a437d5bfff73113da2e0eea949ab288448184deacb64001ee4f5165c"}
Sep 19 11:34:29.711409 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.711329    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8" event={"ID":"c92467c1-64b1-4516-bf34-4d49a767711a","Type":"ContainerStarted","Data":"3618c8e3bf92992fcd1f6e1df3778d1e8a8dd38543f4a9e3b5dc912648a1c0d7"}
Sep 19 11:34:29.712887 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.712867    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58" event={"ID":"99dfbfd8-d697-427c-bc84-0932af7c82ad","Type":"ContainerStarted","Data":"d89f04bb5d02bad83546ea92c25cd2e8dfe1e028396069d685a53852081182a3"}
Sep 19 11:34:29.714448 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.714426    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-98pk7" event={"ID":"6976d909-6428-474c-8e7c-0c012b8e23f8","Type":"ContainerStarted","Data":"21206a13911b460289f4f9b486d1a3c602ffcee90d8b7b83412bd4ee4c0e6b5e"}
Sep 19 11:34:29.715828 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.715804    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zj5bk" event={"ID":"af32c67c-4404-440d-986f-ec4663e165e6","Type":"ContainerStarted","Data":"71eef53b5df57b6774b308fd85b1f82cce5941fcb8009f9fdc6661d2bd4bb9e3"}
Sep 19 11:34:29.717609 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.717584    2559 generic.go:356] "Generic (PLEG): container finished" podID="20b32973-0255-40ad-a150-fa451be068ec" containerID="3d429588662308253cd6c0570c21f9d2ba805136a103f090b56761a766ab481f" exitCode=0
Sep 19 11:34:29.717707 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.717662    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bd7pv" event={"ID":"20b32973-0255-40ad-a150-fa451be068ec","Type":"ContainerDied","Data":"3d429588662308253cd6c0570c21f9d2ba805136a103f090b56761a766ab481f"}
Sep 19 11:34:29.719965 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.719941    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2jpjr" event={"ID":"4002479e-43d6-4fbb-9e74-f9a9d0c02c9c","Type":"ContainerStarted","Data":"1dcdc1b1b9d8dc000380553abd7f60cf8660e1427c480fc50f923b89307064bd"}
Sep 19 11:34:29.721481 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.721458    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mxrb8" event={"ID":"3c490d87-bfde-4aa7-b297-e71e3f1c8056","Type":"ContainerStarted","Data":"6bacca6cd935c2aeb43080e35e80fb0bf3777cffc4fa6708ffb9f5fbc706c8ac"}
Sep 19 11:34:29.723074 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.723035    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fjmwn" podStartSLOduration=2.9309298459999997 podStartE2EDuration="20.723022116s" totalImagesPullingTime="17.79209227s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 11:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:34:29.722760573 +0000 UTC m=+21.645073454" watchObservedRunningTime="2026-09-19 11:34:29.723022116 +0000 UTC m=+21.645334997"
Sep 19 11:34:29.723146 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.723102    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-136.ec2.internal" podStartSLOduration=20.723098199 podStartE2EDuration="20.723098199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 11:34:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:34:13.689361424 +0000 UTC m=+5.611674307" watchObservedRunningTime="2026-09-19 11:34:29.723098199 +0000 UTC m=+21.645411080"
Sep 19 11:34:29.738413 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.738373    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zj5bk" podStartSLOduration=3.926682059 podStartE2EDuration="21.738358048s" totalImagesPullingTime="17.811675989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 11:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:34:29.737771486 +0000 UTC m=+21.660084379" watchObservedRunningTime="2026-09-19 11:34:29.738358048 +0000 UTC m=+21.660670929"
Sep 19 11:34:29.754123 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.754104    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mxrb8"
Sep 19 11:34:29.754679 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.754658    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mxrb8"
Sep 19 11:34:29.772489 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.772448    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-98pk7" podStartSLOduration=3.910104949 podStartE2EDuration="21.772435843s" totalImagesPullingTime="17.862330894s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 11:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:34:29.772169392 +0000 UTC m=+21.694482277" watchObservedRunningTime="2026-09-19 11:34:29.772435843 +0000 UTC m=+21.694748724"
Sep 19 11:34:29.800069 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.800031    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2jpjr" podStartSLOduration=4.00232726 podStartE2EDuration="21.800016586s" totalImagesPullingTime="17.797689326s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 11:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:34:29.786476808 +0000 UTC m=+21.708789688" watchObservedRunningTime="2026-09-19 11:34:29.800016586 +0000 UTC m=+21.722329445"
Sep 19 11:34:29.800596 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:29.800568    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mxrb8" podStartSLOduration=7.964413761 podStartE2EDuration="21.800559132s" totalImagesPullingTime="13.836145371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:34:29.799990894 +0000 UTC m=+21.722303776" watchObservedRunningTime="2026-09-19 11:34:29.800559132 +0000 UTC m=+21.722872012"
Sep 19 11:34:30.589546 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:30.589298    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:30.589546 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:30.589509    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:30.589893 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:30.589298    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:30.589893 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:30.589595    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:30.589893 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:30.589655    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:30.589893 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:30.589724    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:30.725234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:30.725202    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9tfzm" event={"ID":"d3ad7990-91f1-43ee-b69b-e1f4f04f37e7","Type":"ContainerStarted","Data":"4584790b5914f1565a845249030668517f64afcb80de4f85bf6f79894f1890bc"}
Sep 19 11:34:30.728815 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:30.728787    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8" event={"ID":"c92467c1-64b1-4516-bf34-4d49a767711a","Type":"ContainerStarted","Data":"f0810023ada8cc3f0593bccf26d81f0c5185710e57e37ffea39114318bfef7ae"}
Sep 19 11:34:30.728950 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:30.728870    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8" event={"ID":"c92467c1-64b1-4516-bf34-4d49a767711a","Type":"ContainerStarted","Data":"1388267e075b1651228510a9744f7ff2831a88fd5da25e768873adeb1272983a"}
Sep 19 11:34:30.729541 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:30.729517    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mxrb8"
Sep 19 11:34:30.730019 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:30.730004    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mxrb8"
Sep 19 11:34:30.739862 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:30.739817    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9tfzm" podStartSLOduration=4.9435664280000005 podStartE2EDuration="22.739804344s" totalImagesPullingTime="17.796237916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 11:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:34:30.739434239 +0000 UTC m=+22.661747131" watchObservedRunningTime="2026-09-19 11:34:30.739804344 +0000 UTC m=+22.662117216"
Sep 19 11:34:30.990086 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:30.990066    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 19 11:34:31.545664 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:31.545430    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-19T11:34:30.990079738Z","UUID":"11fa7410-7c99-461d-8fd6-4c93636071aa","Handler":null,"Name":"","Endpoint":""}
Sep 19 11:34:31.547092 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:31.547071    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 19 11:34:31.547230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:31.547106    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 19 11:34:31.738343 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:31.738308    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58" event={"ID":"99dfbfd8-d697-427c-bc84-0932af7c82ad","Type":"ContainerStarted","Data":"30d9448450ee3116533aa86ade767799156d00dda6bac245a542d4f577bb235c"}
Sep 19 11:34:32.589489 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:32.589401    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:32.589489 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:32.589485    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:32.589698 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:32.589504    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:32.589698 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:32.589613    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:32.589962 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:32.589928    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:32.590086 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:32.590029    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:32.743116 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:32.743083    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58" event={"ID":"99dfbfd8-d697-427c-bc84-0932af7c82ad","Type":"ContainerStarted","Data":"828f94876a8a3bb61e2099996c804dbc45416148473fbbfd2f5f012b641d7b0b"}
Sep 19 11:34:32.747011 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:32.746980    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8" event={"ID":"c92467c1-64b1-4516-bf34-4d49a767711a","Type":"ContainerStarted","Data":"fb6753aed9cdae320b41349ca57e0f2da706d9b990c1a826c59edacf47130ac9"}
Sep 19 11:34:32.762058 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:32.762016    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2dj58" podStartSLOduration=4.508576668 podStartE2EDuration="24.762003463s" totalImagesPullingTime="20.253426795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 11:34:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:34:32.761440546 +0000 UTC m=+24.683753428" watchObservedRunningTime="2026-09-19 11:34:32.762003463 +0000 UTC m=+24.684316348"
Sep 19 11:34:33.873134 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:33.873104    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fjmwn_6ed690d2-ca31-447d-bda0-ee83ae9efc22/dns-node-resolver/0.log"
Sep 19 11:34:34.589438 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:34.589413    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:34.589568 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:34.589446    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:34.589568 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:34.589413    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:34.589568 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:34.589525    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:34.589722 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:34.589615    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:34.589722 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:34.589701    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:34.756250 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:34.755428    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8" event={"ID":"c92467c1-64b1-4516-bf34-4d49a767711a","Type":"ContainerStarted","Data":"02eb06d6d32a8da8e302a8da9c95bfc194c955a84b545687e89ea8a1225bfec8"}
Sep 19 11:34:34.756250 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:34.755764    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:34.772949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:34.772919    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:34.821028 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:34.820987    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8" podStartSLOduration=8.962209871 podStartE2EDuration="26.820972488s" totalImagesPullingTime="17.858762617s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 11:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:34:34.794100743 +0000 UTC m=+26.716413623" watchObservedRunningTime="2026-09-19 11:34:34.820972488 +0000 UTC m=+26.743285370"
Sep 19 11:34:34.845348 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:34.845280    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2jpjr_4002479e-43d6-4fbb-9e74-f9a9d0c02c9c/node-ca/0.log"
Sep 19 11:34:35.642891 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:35.642862    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:35.759238 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:35.759205    2559 generic.go:356] "Generic (PLEG): container finished" podID="20b32973-0255-40ad-a150-fa451be068ec" containerID="e5a1e8cb86259f5585172027c103f5d70447bfde61b9f88286bac4446a730332" exitCode=0
Sep 19 11:34:35.759397 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:35.759292    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bd7pv" event={"ID":"20b32973-0255-40ad-a150-fa451be068ec","Type":"ContainerDied","Data":"e5a1e8cb86259f5585172027c103f5d70447bfde61b9f88286bac4446a730332"}
Sep 19 11:34:35.759758 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:35.759730    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:35.775726 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:35.775702    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:34:36.589603 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:36.589571    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:36.589841 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:36.589570    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:36.589841 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:36.589677    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:36.589841 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:36.589582    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:36.589841 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:36.589768    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:36.589986 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:36.589841    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:37.768001 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:37.767969    2559 generic.go:356] "Generic (PLEG): container finished" podID="20b32973-0255-40ad-a150-fa451be068ec" containerID="e6468a508f428fb1e8aa7914dbd8efcb4feb57e84445f5bb78eab28a7d03241d" exitCode=0
Sep 19 11:34:37.768568 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:37.768054    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bd7pv" event={"ID":"20b32973-0255-40ad-a150-fa451be068ec","Type":"ContainerDied","Data":"e6468a508f428fb1e8aa7914dbd8efcb4feb57e84445f5bb78eab28a7d03241d"}
Sep 19 11:34:38.590282 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:38.590252    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:38.590439 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:38.590353    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:38.590439 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:38.590426    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:38.590539 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:38.590515    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:38.590539 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:38.590531    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:38.590657 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:38.590634    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:38.772935 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:38.772904    2559 generic.go:356] "Generic (PLEG): container finished" podID="20b32973-0255-40ad-a150-fa451be068ec" containerID="32024ba7107191e44d95fa9aa9289abc345a3271a7cf4062598f7ee986672fb8" exitCode=0
Sep 19 11:34:38.773415 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:38.772950    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bd7pv" event={"ID":"20b32973-0255-40ad-a150-fa451be068ec","Type":"ContainerDied","Data":"32024ba7107191e44d95fa9aa9289abc345a3271a7cf4062598f7ee986672fb8"}
Sep 19 11:34:40.589517 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:40.589481    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:40.589517 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:40.589524    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:40.590030 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:40.589644    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:40.590030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:40.589703    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:40.590030 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:40.589782    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:40.590030 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:40.589833    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:42.315782 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:42.315743    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:42.315782 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:42.315787    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs\") pod \"network-metrics-daemon-fm8xq\" (UID: \"50a0f2dc-b541-449e-8e88-8f871f1b30b5\") " pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:42.316302 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:42.315819    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brnlp\" (UniqueName: \"kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp\") pod \"network-check-target-lslq5\" (UID: \"9ba68c1a-f003-473e-9d35-d86153d8c160\") " pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:42.316302 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:42.315898    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 11:34:42.316302 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:42.315908    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 11:34:42.316302 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:42.315934    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 11:34:42.316302 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:42.315956    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 11:34:42.316302 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:42.315961    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret podName:fdf9b2d4-b351-4020-8ee5-4b00b841066d nodeName:}" failed. No retries permitted until 2026-09-19 11:35:14.315944149 +0000 UTC m=+66.238257012 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret") pod "global-pull-secret-syncer-w6hmz" (UID: "fdf9b2d4-b351-4020-8ee5-4b00b841066d") : object "kube-system"/"original-pull-secret" not registered
Sep 19 11:34:42.316302 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:42.315967    2559 projected.go:196] Error preparing data for projected volume kube-api-access-brnlp for pod openshift-network-diagnostics/network-check-target-lslq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 11:34:42.316302 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:42.315977    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs podName:50a0f2dc-b541-449e-8e88-8f871f1b30b5 nodeName:}" failed. No retries permitted until 2026-09-19 11:35:14.3159703 +0000 UTC m=+66.238283159 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs") pod "network-metrics-daemon-fm8xq" (UID: "50a0f2dc-b541-449e-8e88-8f871f1b30b5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 11:34:42.316302 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:42.316003    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp podName:9ba68c1a-f003-473e-9d35-d86153d8c160 nodeName:}" failed. No retries permitted until 2026-09-19 11:35:14.315989668 +0000 UTC m=+66.238302532 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-brnlp" (UniqueName: "kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp") pod "network-check-target-lslq5" (UID: "9ba68c1a-f003-473e-9d35-d86153d8c160") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 11:34:42.589051 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:42.588973    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:42.589051 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:42.589010    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:42.589253 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:42.588974    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:42.589253 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:42.589113    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:42.589253 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:42.589167    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:42.589253 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:42.589234    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:43.787798 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:43.787766    2559 generic.go:356] "Generic (PLEG): container finished" podID="20b32973-0255-40ad-a150-fa451be068ec" containerID="9fc3f442b84d8795d17bc42df8d55529594593d915b4e970af5b93b5abf21b9d" exitCode=0
Sep 19 11:34:43.788180 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:43.787822    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bd7pv" event={"ID":"20b32973-0255-40ad-a150-fa451be068ec","Type":"ContainerDied","Data":"9fc3f442b84d8795d17bc42df8d55529594593d915b4e970af5b93b5abf21b9d"}
Sep 19 11:34:44.588872 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:44.588839    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:44.588872 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:44.588863    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:44.589117 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:44.588869    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:44.589117 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:44.588950    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:44.589117 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:44.589025    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:44.589290 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:44.589117    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:44.793073 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:44.793044    2559 generic.go:356] "Generic (PLEG): container finished" podID="20b32973-0255-40ad-a150-fa451be068ec" containerID="a846c905f71f376e3e0b192b6d8a38420b410e0a1d807a540dd783369696d0dd" exitCode=0
Sep 19 11:34:44.793474 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:44.793100    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bd7pv" event={"ID":"20b32973-0255-40ad-a150-fa451be068ec","Type":"ContainerDied","Data":"a846c905f71f376e3e0b192b6d8a38420b410e0a1d807a540dd783369696d0dd"}
Sep 19 11:34:45.798695 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:45.798517    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bd7pv" event={"ID":"20b32973-0255-40ad-a150-fa451be068ec","Type":"ContainerStarted","Data":"4f277478f12364cb516490a60a83fafc0fa67b653db6a9f22fe0ac0b48276382"}
Sep 19 11:34:45.817117 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:45.817074    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bd7pv" podStartSLOduration=16.074888341 podStartE2EDuration="37.817061334s" totalImagesPullingTime="21.742172993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 11:34:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:34:45.815990925 +0000 UTC m=+37.738303805" watchObservedRunningTime="2026-09-19 11:34:45.817061334 +0000 UTC m=+37.739374215"
Sep 19 11:34:46.588702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:46.588668    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:46.588888 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:46.588674    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:46.588888 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:46.588783    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:46.588888 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:46.588674    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:46.588888 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:46.588873    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:46.589105 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:46.588943    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:48.589278 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:48.589252    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:48.589683 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:48.589332    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:48.589683 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:48.589364    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:48.589683 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:48.589511    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:48.589683 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:48.589548    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:48.589683 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:48.589648    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:50.588806 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:50.588769    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:50.589259 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:50.588769    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:50.589259 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:50.588900    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:50.589259 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:50.588767    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:50.589259 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:50.588974    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:50.589259 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:50.589038    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:52.589422 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:52.589391    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:52.589422 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:52.589416    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:52.589930 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:52.589478    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:52.589930 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:52.589392    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:52.589930 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:52.589574    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:52.589930 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:52.589660    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:53.635711 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:53.635642    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w6hmz"]
Sep 19 11:34:53.636168 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:53.635756    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:53.636168 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:53.635855    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:53.639547 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:53.639519    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fm8xq"]
Sep 19 11:34:53.639689 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:53.639620    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:53.639752 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:53.639718    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:53.640215 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:53.640152    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lslq5"]
Sep 19 11:34:53.640332 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:53.640248    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:53.640396 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:53.640328    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:55.589287 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:55.589181    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:55.589687 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:55.589204    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:55.589687 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:55.589313    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:55.589687 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:55.589328    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:55.589687 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:55.589387    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:55.589687 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:55.589481    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:57.588896 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:57.588867    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:57.589255 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:57.588964    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:57.589255 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:57.588976    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lslq5" podUID="9ba68c1a-f003-473e-9d35-d86153d8c160"
Sep 19 11:34:57.589255 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:57.588985    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:57.589255 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:57.589063    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fm8xq" podUID="50a0f2dc-b541-449e-8e88-8f871f1b30b5"
Sep 19 11:34:57.589255 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:34:57.589124    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w6hmz" podUID="fdf9b2d4-b351-4020-8ee5-4b00b841066d"
Sep 19 11:34:58.362781 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.362622    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-136.ec2.internal" event="NodeReady"
Sep 19 11:34:58.362979 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.362854    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 19 11:34:58.410155 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.410129    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jkjln"]
Sep 19 11:34:58.414724 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.414703    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nfdpq"]
Sep 19 11:34:58.414860 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.414842    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jkjln"
Sep 19 11:34:58.417863 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.417462    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mvj2p\""
Sep 19 11:34:58.417863 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.417475    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 19 11:34:58.417863 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.417477    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 19 11:34:58.421927 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.421897    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.424174 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.424151    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jkjln"]
Sep 19 11:34:58.424306 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.424292    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-g979l\""
Sep 19 11:34:58.424306 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.424301    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 19 11:34:58.424452 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.424430    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 19 11:34:58.424752 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.424737    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 19 11:34:58.425012 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.424994    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 19 11:34:58.427140 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.427121    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nfdpq"]
Sep 19 11:34:58.438202 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.438166    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6d57e236-44c2-4acc-8ab0-342a59daf9c9-crio-socket\") pod \"insights-runtime-extractor-nfdpq\" (UID: \"6d57e236-44c2-4acc-8ab0-342a59daf9c9\") " pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.438280 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.438238    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d57e236-44c2-4acc-8ab0-342a59daf9c9-data-volume\") pod \"insights-runtime-extractor-nfdpq\" (UID: \"6d57e236-44c2-4acc-8ab0-342a59daf9c9\") " pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.438280 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.438257    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q5pq\" (UniqueName: \"kubernetes.io/projected/aea6548e-5e35-48f8-b05b-813264f85e5d-kube-api-access-9q5pq\") pod \"dns-default-jkjln\" (UID: \"aea6548e-5e35-48f8-b05b-813264f85e5d\") " pod="openshift-dns/dns-default-jkjln"
Sep 19 11:34:58.438280 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.438275    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r442\" (UniqueName: \"kubernetes.io/projected/6d57e236-44c2-4acc-8ab0-342a59daf9c9-kube-api-access-7r442\") pod \"insights-runtime-extractor-nfdpq\" (UID: \"6d57e236-44c2-4acc-8ab0-342a59daf9c9\") " pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.438377 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.438289    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aea6548e-5e35-48f8-b05b-813264f85e5d-config-volume\") pod \"dns-default-jkjln\" (UID: \"aea6548e-5e35-48f8-b05b-813264f85e5d\") " pod="openshift-dns/dns-default-jkjln"
Sep 19 11:34:58.438377 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.438341    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aea6548e-5e35-48f8-b05b-813264f85e5d-tmp-dir\") pod \"dns-default-jkjln\" (UID: \"aea6548e-5e35-48f8-b05b-813264f85e5d\") " pod="openshift-dns/dns-default-jkjln"
Sep 19 11:34:58.438377 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.438373    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aea6548e-5e35-48f8-b05b-813264f85e5d-metrics-tls\") pod \"dns-default-jkjln\" (UID: \"aea6548e-5e35-48f8-b05b-813264f85e5d\") " pod="openshift-dns/dns-default-jkjln"
Sep 19 11:34:58.438465 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.438410    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d57e236-44c2-4acc-8ab0-342a59daf9c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nfdpq\" (UID: \"6d57e236-44c2-4acc-8ab0-342a59daf9c9\") " pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.438465 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.438428    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d57e236-44c2-4acc-8ab0-342a59daf9c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nfdpq\" (UID: \"6d57e236-44c2-4acc-8ab0-342a59daf9c9\") " pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.514263 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.514238    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pz68l"]
Sep 19 11:34:58.527964 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.527947    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pz68l"]
Sep 19 11:34:58.528074 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.528057    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pz68l"
Sep 19 11:34:58.530466 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.530450    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 19 11:34:58.530984 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.530971    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 19 11:34:58.531055 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.530981    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-t9jnr\""
Sep 19 11:34:58.531288 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.531272    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 19 11:34:58.538647 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.538629    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d57e236-44c2-4acc-8ab0-342a59daf9c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nfdpq\" (UID: \"6d57e236-44c2-4acc-8ab0-342a59daf9c9\") " pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.538722 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.538653    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3-cert\") pod \"ingress-canary-pz68l\" (UID: \"3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3\") " pod="openshift-ingress-canary/ingress-canary-pz68l"
Sep 19 11:34:58.538722 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.538671    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d57e236-44c2-4acc-8ab0-342a59daf9c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nfdpq\" (UID: \"6d57e236-44c2-4acc-8ab0-342a59daf9c9\") " pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.538722 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.538697    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6d57e236-44c2-4acc-8ab0-342a59daf9c9-crio-socket\") pod \"insights-runtime-extractor-nfdpq\" (UID: \"6d57e236-44c2-4acc-8ab0-342a59daf9c9\") " pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.538830 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.538721    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qprf8\" (UniqueName: \"kubernetes.io/projected/3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3-kube-api-access-qprf8\") pod \"ingress-canary-pz68l\" (UID: \"3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3\") " pod="openshift-ingress-canary/ingress-canary-pz68l"
Sep 19 11:34:58.538830 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.538766    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d57e236-44c2-4acc-8ab0-342a59daf9c9-data-volume\") pod \"insights-runtime-extractor-nfdpq\" (UID: \"6d57e236-44c2-4acc-8ab0-342a59daf9c9\") " pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.538830 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.538787    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q5pq\" (UniqueName: \"kubernetes.io/projected/aea6548e-5e35-48f8-b05b-813264f85e5d-kube-api-access-9q5pq\") pod \"dns-default-jkjln\" (UID: \"aea6548e-5e35-48f8-b05b-813264f85e5d\") " pod="openshift-dns/dns-default-jkjln"
Sep 19 11:34:58.538830 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.538810    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r442\" (UniqueName: \"kubernetes.io/projected/6d57e236-44c2-4acc-8ab0-342a59daf9c9-kube-api-access-7r442\") pod \"insights-runtime-extractor-nfdpq\" (UID: \"6d57e236-44c2-4acc-8ab0-342a59daf9c9\") " pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.539013 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.538831    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aea6548e-5e35-48f8-b05b-813264f85e5d-config-volume\") pod \"dns-default-jkjln\" (UID: \"aea6548e-5e35-48f8-b05b-813264f85e5d\") " pod="openshift-dns/dns-default-jkjln"
Sep 19 11:34:58.539013 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.538851    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aea6548e-5e35-48f8-b05b-813264f85e5d-tmp-dir\") pod \"dns-default-jkjln\" (UID: \"aea6548e-5e35-48f8-b05b-813264f85e5d\") " pod="openshift-dns/dns-default-jkjln"
Sep 19 11:34:58.539013 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.538883    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aea6548e-5e35-48f8-b05b-813264f85e5d-metrics-tls\") pod \"dns-default-jkjln\" (UID: \"aea6548e-5e35-48f8-b05b-813264f85e5d\") " pod="openshift-dns/dns-default-jkjln"
Sep 19 11:34:58.539013 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.538903    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6d57e236-44c2-4acc-8ab0-342a59daf9c9-crio-socket\") pod \"insights-runtime-extractor-nfdpq\" (UID: \"6d57e236-44c2-4acc-8ab0-342a59daf9c9\") " pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.539171 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.539130    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d57e236-44c2-4acc-8ab0-342a59daf9c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nfdpq\" (UID: \"6d57e236-44c2-4acc-8ab0-342a59daf9c9\") " pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.539316 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.539298    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aea6548e-5e35-48f8-b05b-813264f85e5d-tmp-dir\") pod \"dns-default-jkjln\" (UID: \"aea6548e-5e35-48f8-b05b-813264f85e5d\") " pod="openshift-dns/dns-default-jkjln"
Sep 19 11:34:58.539455 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.539436    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aea6548e-5e35-48f8-b05b-813264f85e5d-config-volume\") pod \"dns-default-jkjln\" (UID: \"aea6548e-5e35-48f8-b05b-813264f85e5d\") " pod="openshift-dns/dns-default-jkjln"
Sep 19 11:34:58.543099 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.543079    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aea6548e-5e35-48f8-b05b-813264f85e5d-metrics-tls\") pod \"dns-default-jkjln\" (UID: \"aea6548e-5e35-48f8-b05b-813264f85e5d\") " pod="openshift-dns/dns-default-jkjln"
Sep 19 11:34:58.547663 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.547646    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d57e236-44c2-4acc-8ab0-342a59daf9c9-data-volume\") pod \"insights-runtime-extractor-nfdpq\" (UID: \"6d57e236-44c2-4acc-8ab0-342a59daf9c9\") " pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.549571 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.549555    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d57e236-44c2-4acc-8ab0-342a59daf9c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nfdpq\" (UID: \"6d57e236-44c2-4acc-8ab0-342a59daf9c9\") " pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.555307 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.555289    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r442\" (UniqueName: \"kubernetes.io/projected/6d57e236-44c2-4acc-8ab0-342a59daf9c9-kube-api-access-7r442\") pod \"insights-runtime-extractor-nfdpq\" (UID: \"6d57e236-44c2-4acc-8ab0-342a59daf9c9\") " pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.558582 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.558561    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q5pq\" (UniqueName: \"kubernetes.io/projected/aea6548e-5e35-48f8-b05b-813264f85e5d-kube-api-access-9q5pq\") pod \"dns-default-jkjln\" (UID: \"aea6548e-5e35-48f8-b05b-813264f85e5d\") " pod="openshift-dns/dns-default-jkjln"
Sep 19 11:34:58.640124 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.640071    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3-cert\") pod \"ingress-canary-pz68l\" (UID: \"3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3\") " pod="openshift-ingress-canary/ingress-canary-pz68l"
Sep 19 11:34:58.640124 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.640108    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qprf8\" (UniqueName: \"kubernetes.io/projected/3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3-kube-api-access-qprf8\") pod \"ingress-canary-pz68l\" (UID: \"3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3\") " pod="openshift-ingress-canary/ingress-canary-pz68l"
Sep 19 11:34:58.642284 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.642269    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3-cert\") pod \"ingress-canary-pz68l\" (UID: \"3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3\") " pod="openshift-ingress-canary/ingress-canary-pz68l"
Sep 19 11:34:58.648207 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.648175    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qprf8\" (UniqueName: \"kubernetes.io/projected/3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3-kube-api-access-qprf8\") pod \"ingress-canary-pz68l\" (UID: \"3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3\") " pod="openshift-ingress-canary/ingress-canary-pz68l"
Sep 19 11:34:58.727127 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.727091    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jkjln"
Sep 19 11:34:58.742519 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.742489    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nfdpq"
Sep 19 11:34:58.836847 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:58.836623    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pz68l"
Sep 19 11:34:59.045045 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.045017    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jkjln"]
Sep 19 11:34:59.048460 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.048438    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nfdpq"]
Sep 19 11:34:59.048856 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:59.048836    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaea6548e_5e35_48f8_b05b_813264f85e5d.slice/crio-6b783492136d4cdd7d676d9a3d07aed9cf30250a0a8afe0247c40433831c4384 WatchSource:0}: Error finding container 6b783492136d4cdd7d676d9a3d07aed9cf30250a0a8afe0247c40433831c4384: Status 404 returned error can't find the container with id 6b783492136d4cdd7d676d9a3d07aed9cf30250a0a8afe0247c40433831c4384
Sep 19 11:34:59.052040 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:59.052016    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d57e236_44c2_4acc_8ab0_342a59daf9c9.slice/crio-378498e99abd46a12779a778903e51c185ef5f74bdfd152e0315c8a58baedc58 WatchSource:0}: Error finding container 378498e99abd46a12779a778903e51c185ef5f74bdfd152e0315c8a58baedc58: Status 404 returned error can't find the container with id 378498e99abd46a12779a778903e51c185ef5f74bdfd152e0315c8a58baedc58
Sep 19 11:34:59.126898 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.126780    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pz68l"]
Sep 19 11:34:59.129995 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:34:59.129974    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c4b91af_0fb0_4cf6_b0bf_08ef43e9b4d3.slice/crio-c1e45c847af895124bb1ba03329d4c06c4f95b58cb93539f7b223602d1921c92 WatchSource:0}: Error finding container c1e45c847af895124bb1ba03329d4c06c4f95b58cb93539f7b223602d1921c92: Status 404 returned error can't find the container with id c1e45c847af895124bb1ba03329d4c06c4f95b58cb93539f7b223602d1921c92
Sep 19 11:34:59.589321 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.589288    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:34:59.589321 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.589318    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:34:59.589623 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.589288    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:34:59.592050 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.592027    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 19 11:34:59.592326 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.592309    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6c6f7\""
Sep 19 11:34:59.592606 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.592588    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 11:34:59.592863 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.592839    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 11:34:59.593057 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.593042    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 19 11:34:59.593315 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.593299    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lhwvf\""
Sep 19 11:34:59.831434 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.831357    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jkjln" event={"ID":"aea6548e-5e35-48f8-b05b-813264f85e5d","Type":"ContainerStarted","Data":"6b783492136d4cdd7d676d9a3d07aed9cf30250a0a8afe0247c40433831c4384"}
Sep 19 11:34:59.833023 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.832989    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pz68l" event={"ID":"3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3","Type":"ContainerStarted","Data":"c1e45c847af895124bb1ba03329d4c06c4f95b58cb93539f7b223602d1921c92"}
Sep 19 11:34:59.836130 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.836100    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nfdpq" event={"ID":"6d57e236-44c2-4acc-8ab0-342a59daf9c9","Type":"ContainerStarted","Data":"2fd42288ee38d82e8093cea9ea53394ef81c0b32a57eb05526a86b299ff0a146"}
Sep 19 11:34:59.836271 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:34:59.836155    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nfdpq" event={"ID":"6d57e236-44c2-4acc-8ab0-342a59daf9c9","Type":"ContainerStarted","Data":"378498e99abd46a12779a778903e51c185ef5f74bdfd152e0315c8a58baedc58"}
Sep 19 11:35:00.841586 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:00.841518    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nfdpq" event={"ID":"6d57e236-44c2-4acc-8ab0-342a59daf9c9","Type":"ContainerStarted","Data":"6ba9fded97bb48f2be61afe19bcd8fd8d3d4598afebd9be0b1810d95d7a0b10e"}
Sep 19 11:35:00.846552 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:00.846475    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jkjln" event={"ID":"aea6548e-5e35-48f8-b05b-813264f85e5d","Type":"ContainerStarted","Data":"aee04755444ecc931d67f47f910e3ab98fb4c48602d21d363f82ab5dc1234ead"}
Sep 19 11:35:01.851790 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:01.851696    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jkjln" event={"ID":"aea6548e-5e35-48f8-b05b-813264f85e5d","Type":"ContainerStarted","Data":"e0cdf40815578a04894f16b0a0d273b37ed7f78805246b1c78d5c67a481e97c1"}
Sep 19 11:35:01.852246 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:01.851818    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jkjln"
Sep 19 11:35:01.867518 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:01.867468    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jkjln" podStartSLOduration=2.2099549290000002 podStartE2EDuration="3.867450823s" totalImagesPullingTime="1.657495894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:34:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:01.866126016 +0000 UTC m=+53.788438897" watchObservedRunningTime="2026-09-19 11:35:01.867450823 +0000 UTC m=+53.789763704"
Sep 19 11:35:02.858162 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:02.857930    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pz68l" event={"ID":"3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3","Type":"ContainerStarted","Data":"3e48806e76a066aad213e47cee74f2f13bbfdb9033a9f3b335088267a0222a01"}
Sep 19 11:35:02.860477 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:02.860451    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nfdpq" event={"ID":"6d57e236-44c2-4acc-8ab0-342a59daf9c9","Type":"ContainerStarted","Data":"a135ee5a32c66e6508830ea80574d1faaca2d46fe1f516dded296f7b87bcbb92"}
Sep 19 11:35:02.875713 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:02.875674    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pz68l" podStartSLOduration=1.9399298950000001 podStartE2EDuration="4.875662466s" totalImagesPullingTime="2.935732571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:34:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:02.875140143 +0000 UTC m=+54.797453024" watchObservedRunningTime="2026-09-19 11:35:02.875662466 +0000 UTC m=+54.797975347"
Sep 19 11:35:02.896861 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:02.896743    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nfdpq" podStartSLOduration=2.097762809 podStartE2EDuration="4.896727034s" totalImagesPullingTime="2.798964225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 11:34:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:02.89644927 +0000 UTC m=+54.818762143" watchObservedRunningTime="2026-09-19 11:35:02.896727034 +0000 UTC m=+54.819039892"
Sep 19 11:35:07.779344 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:07.779312    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mx5t8"
Sep 19 11:35:11.863041 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:11.863007    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jkjln"
Sep 19 11:35:14.347592 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.347555    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:35:14.348035 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.347598    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs\") pod \"network-metrics-daemon-fm8xq\" (UID: \"50a0f2dc-b541-449e-8e88-8f871f1b30b5\") " pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:35:14.348035 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.347631    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brnlp\" (UniqueName: \"kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp\") pod \"network-check-target-lslq5\" (UID: \"9ba68c1a-f003-473e-9d35-d86153d8c160\") " pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:35:14.349855 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.349835    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 11:35:14.349939 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.349835    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 11:35:14.350652 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.350634    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 19 11:35:14.360335 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.360316    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 19 11:35:14.362156 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.362135    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a0f2dc-b541-449e-8e88-8f871f1b30b5-metrics-certs\") pod \"network-metrics-daemon-fm8xq\" (UID: \"50a0f2dc-b541-449e-8e88-8f871f1b30b5\") " pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:35:14.362254 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.362210    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fdf9b2d4-b351-4020-8ee5-4b00b841066d-original-pull-secret\") pod \"global-pull-secret-syncer-w6hmz\" (UID: \"fdf9b2d4-b351-4020-8ee5-4b00b841066d\") " pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:35:14.371250 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.371232    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brnlp\" (UniqueName: \"kubernetes.io/projected/9ba68c1a-f003-473e-9d35-d86153d8c160-kube-api-access-brnlp\") pod \"network-check-target-lslq5\" (UID: \"9ba68c1a-f003-473e-9d35-d86153d8c160\") " pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:35:14.606952 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.606873    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lhwvf\""
Sep 19 11:35:14.611642 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.611628    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w6hmz"
Sep 19 11:35:14.615048 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.615029    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:35:14.619588 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.619571    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6c6f7\""
Sep 19 11:35:14.629454 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.628311    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fm8xq"
Sep 19 11:35:14.889477 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.889295    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lslq5"]
Sep 19 11:35:14.891644 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.891619    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w6hmz"]
Sep 19 11:35:14.892228 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:35:14.892206    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ba68c1a_f003_473e_9d35_d86153d8c160.slice/crio-1e056c18a7d8f1762c8b5f38ae56e8736f3ca499fa3e0ef017a23c11d2b67ba5 WatchSource:0}: Error finding container 1e056c18a7d8f1762c8b5f38ae56e8736f3ca499fa3e0ef017a23c11d2b67ba5: Status 404 returned error can't find the container with id 1e056c18a7d8f1762c8b5f38ae56e8736f3ca499fa3e0ef017a23c11d2b67ba5
Sep 19 11:35:14.895333 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:35:14.895313    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdf9b2d4_b351_4020_8ee5_4b00b841066d.slice/crio-4da7b91f4f502afe79a0be41519c8b845529e46f05563032b9a53d8b9794a942 WatchSource:0}: Error finding container 4da7b91f4f502afe79a0be41519c8b845529e46f05563032b9a53d8b9794a942: Status 404 returned error can't find the container with id 4da7b91f4f502afe79a0be41519c8b845529e46f05563032b9a53d8b9794a942
Sep 19 11:35:14.900206 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.900171    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fm8xq"]
Sep 19 11:35:14.900840 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.900816    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w6hmz" event={"ID":"fdf9b2d4-b351-4020-8ee5-4b00b841066d","Type":"ContainerStarted","Data":"4da7b91f4f502afe79a0be41519c8b845529e46f05563032b9a53d8b9794a942"}
Sep 19 11:35:14.902005 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:14.901984    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lslq5" event={"ID":"9ba68c1a-f003-473e-9d35-d86153d8c160","Type":"ContainerStarted","Data":"1e056c18a7d8f1762c8b5f38ae56e8736f3ca499fa3e0ef017a23c11d2b67ba5"}
Sep 19 11:35:14.903739 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:35:14.903712    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50a0f2dc_b541_449e_8e88_8f871f1b30b5.slice/crio-0e337c1d2f1ed5a0a1abf434281a2c34b3a6e577e15d165139a7780c1088a634 WatchSource:0}: Error finding container 0e337c1d2f1ed5a0a1abf434281a2c34b3a6e577e15d165139a7780c1088a634: Status 404 returned error can't find the container with id 0e337c1d2f1ed5a0a1abf434281a2c34b3a6e577e15d165139a7780c1088a634
Sep 19 11:35:15.907665 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:15.907599    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fm8xq" event={"ID":"50a0f2dc-b541-449e-8e88-8f871f1b30b5","Type":"ContainerStarted","Data":"0e337c1d2f1ed5a0a1abf434281a2c34b3a6e577e15d165139a7780c1088a634"}
Sep 19 11:35:16.918212 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:16.917759    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fm8xq" event={"ID":"50a0f2dc-b541-449e-8e88-8f871f1b30b5","Type":"ContainerStarted","Data":"c6e2e801780febb7e206670a2f9973d4fc1aa0bed1b5e97cd7a1f0fb42db8176"}
Sep 19 11:35:16.918212 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:16.917851    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fm8xq" event={"ID":"50a0f2dc-b541-449e-8e88-8f871f1b30b5","Type":"ContainerStarted","Data":"c00fdf22fb003f53e873231ead10ad3dba39f8cb9af8db2ff073264df73c1b90"}
Sep 19 11:35:16.933158 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:16.933104    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fm8xq" podStartSLOduration=67.94338107 podStartE2EDuration="1m8.933086119s" totalImagesPullingTime="989.705049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:16.931822267 +0000 UTC m=+68.854135151" watchObservedRunningTime="2026-09-19 11:35:16.933086119 +0000 UTC m=+68.855399002"
Sep 19 11:35:17.861562 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:17.861531    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-lmpmn"]
Sep 19 11:35:17.864692 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:17.864672    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-lmpmn"
Sep 19 11:35:17.866909 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:17.866889    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 19 11:35:17.867012 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:17.866906    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-8jklf\""
Sep 19 11:35:17.872246 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:17.871980    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-lmpmn"]
Sep 19 11:35:17.974889 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:17.974860    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/46543144-422d-406a-a81f-289c42535283-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-lmpmn\" (UID: \"46543144-422d-406a-a81f-289c42535283\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-lmpmn"
Sep 19 11:35:18.075440 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.075404    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/46543144-422d-406a-a81f-289c42535283-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-lmpmn\" (UID: \"46543144-422d-406a-a81f-289c42535283\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-lmpmn"
Sep 19 11:35:18.078060 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.078037    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/46543144-422d-406a-a81f-289c42535283-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-lmpmn\" (UID: \"46543144-422d-406a-a81f-289c42535283\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-lmpmn"
Sep 19 11:35:18.138860 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.138790    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b89cfd9b4-5lfjt"]
Sep 19 11:35:18.141991 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.141971    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.144298 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.144278    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-26r9q\""
Sep 19 11:35:18.144611 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.144575    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 19 11:35:18.144739 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.144714    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 19 11:35:18.144933 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.144919    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 19 11:35:18.145183 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.145166    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 19 11:35:18.145183 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.145183    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 19 11:35:18.145346 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.145290    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 19 11:35:18.145518 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.145501    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 19 11:35:18.152517 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.152499    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b89cfd9b4-5lfjt"]
Sep 19 11:35:18.174880 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.174855    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-lmpmn"
Sep 19 11:35:18.176110 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.176084    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-serving-cert\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.176256 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.176113    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-oauth-serving-cert\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.176256 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.176134    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-oauth-config\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.176256 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.176156    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-service-ca\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.176421 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.176273    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6mjb\" (UniqueName: \"kubernetes.io/projected/cfbe13bc-b73b-4b57-bede-50e74ee50d57-kube-api-access-f6mjb\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.176421 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.176315    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-config\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.277115 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.277076    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6mjb\" (UniqueName: \"kubernetes.io/projected/cfbe13bc-b73b-4b57-bede-50e74ee50d57-kube-api-access-f6mjb\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.277115 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.277115    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-config\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.277369 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.277211    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-serving-cert\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.277369 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.277320    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-oauth-serving-cert\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.277369 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.277355    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-oauth-config\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.277520 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.277385    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-service-ca\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.278061 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.278037    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-service-ca\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.278061 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.278050    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-config\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.278254 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.278056    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-oauth-serving-cert\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.280093 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.280068    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-oauth-config\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.280285 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.280261    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-serving-cert\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.285967 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.285943    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6mjb\" (UniqueName: \"kubernetes.io/projected/cfbe13bc-b73b-4b57-bede-50e74ee50d57-kube-api-access-f6mjb\") pod \"console-b89cfd9b4-5lfjt\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") " pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:18.453405 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:18.453349    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:19.132926 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.132893    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9844b888d-plh7r"]
Sep 19 11:35:19.135941 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.135917    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.143515 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.143438    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 19 11:35:19.144887 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.144832    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9844b888d-plh7r"]
Sep 19 11:35:19.182509 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.182463    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70bed969-3852-4859-9d55-a1546255bf3e-console-serving-cert\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.182509 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.182504    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-trusted-ca-bundle\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.182688 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.182559    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70bed969-3852-4859-9d55-a1546255bf3e-console-oauth-config\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.182688 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.182588    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-console-config\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.182688 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.182615    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rfjk\" (UniqueName: \"kubernetes.io/projected/70bed969-3852-4859-9d55-a1546255bf3e-kube-api-access-9rfjk\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.182688 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.182638    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-service-ca\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.182688 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.182656    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-oauth-serving-cert\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.283378 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.283342    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70bed969-3852-4859-9d55-a1546255bf3e-console-serving-cert\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.283378 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.283380    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-trusted-ca-bundle\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.283603 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.283407    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70bed969-3852-4859-9d55-a1546255bf3e-console-oauth-config\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.284036 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.284010    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-console-config\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.284098 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.284079    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rfjk\" (UniqueName: \"kubernetes.io/projected/70bed969-3852-4859-9d55-a1546255bf3e-kube-api-access-9rfjk\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.284155 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.284126    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-service-ca\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.284658 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.284629    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-trusted-ca-bundle\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.284904 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.284881    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-oauth-serving-cert\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.285260 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.284994    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-service-ca\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.285488 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.285469    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-oauth-serving-cert\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.286678 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.286649    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70bed969-3852-4859-9d55-a1546255bf3e-console-serving-cert\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.288419 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.287662    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70bed969-3852-4859-9d55-a1546255bf3e-console-oauth-config\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.288713 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.288697    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-console-config\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.292656 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.292632    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rfjk\" (UniqueName: \"kubernetes.io/projected/70bed969-3852-4859-9d55-a1546255bf3e-kube-api-access-9rfjk\") pod \"console-9844b888d-plh7r\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") " pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.448107 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.448070    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:19.931620 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.931546    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lslq5" event={"ID":"9ba68c1a-f003-473e-9d35-d86153d8c160","Type":"ContainerStarted","Data":"5e87161159fb92c05e8eb04cbaaa474c4694d4f4488544445a7b4212368262cb"}
Sep 19 11:35:19.931845 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.931738    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:35:19.945866 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:19.945814    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lslq5" podStartSLOduration=67.023181435 podStartE2EDuration="1m11.945799405s" totalImagesPullingTime="4.92261797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:19.944231671 +0000 UTC m=+71.866544554" watchObservedRunningTime="2026-09-19 11:35:19.945799405 +0000 UTC m=+71.868112286"
Sep 19 11:35:20.091088 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:20.091014    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-lmpmn"]
Sep 19 11:35:20.094655 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:35:20.094526    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46543144_422d_406a_a81f_289c42535283.slice/crio-a4081c2d6b8c4159e69651a3d75b9ce945566fd2fbdf8cfad9702e40084b2321 WatchSource:0}: Error finding container a4081c2d6b8c4159e69651a3d75b9ce945566fd2fbdf8cfad9702e40084b2321: Status 404 returned error can't find the container with id a4081c2d6b8c4159e69651a3d75b9ce945566fd2fbdf8cfad9702e40084b2321
Sep 19 11:35:20.095034 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:20.095015    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9844b888d-plh7r"]
Sep 19 11:35:20.097721 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:20.097570    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b89cfd9b4-5lfjt"]
Sep 19 11:35:20.097721 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:35:20.097675    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70bed969_3852_4859_9d55_a1546255bf3e.slice/crio-5ff0127f2c21a82a3480d89394ed131c3e971906104041fe992cae80852cfc63 WatchSource:0}: Error finding container 5ff0127f2c21a82a3480d89394ed131c3e971906104041fe992cae80852cfc63: Status 404 returned error can't find the container with id 5ff0127f2c21a82a3480d89394ed131c3e971906104041fe992cae80852cfc63
Sep 19 11:35:20.100476 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:35:20.100454    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbe13bc_b73b_4b57_bede_50e74ee50d57.slice/crio-e8f2c7a88eab11de1cbce110c2ce242369d58dd4b725db214065e3580f947e83 WatchSource:0}: Error finding container e8f2c7a88eab11de1cbce110c2ce242369d58dd4b725db214065e3580f947e83: Status 404 returned error can't find the container with id e8f2c7a88eab11de1cbce110c2ce242369d58dd4b725db214065e3580f947e83
Sep 19 11:35:20.939483 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:20.939442    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b89cfd9b4-5lfjt" event={"ID":"cfbe13bc-b73b-4b57-bede-50e74ee50d57","Type":"ContainerStarted","Data":"e8f2c7a88eab11de1cbce110c2ce242369d58dd4b725db214065e3580f947e83"}
Sep 19 11:35:20.945505 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:20.945470    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w6hmz" event={"ID":"fdf9b2d4-b351-4020-8ee5-4b00b841066d","Type":"ContainerStarted","Data":"89b9c17259c9ecc3882b0c173e3f8f6d743f0629d205b690b656f90d77572631"}
Sep 19 11:35:20.952260 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:20.952229    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9844b888d-plh7r" event={"ID":"70bed969-3852-4859-9d55-a1546255bf3e","Type":"ContainerStarted","Data":"5ff0127f2c21a82a3480d89394ed131c3e971906104041fe992cae80852cfc63"}
Sep 19 11:35:20.957267 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:20.957244    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-lmpmn" event={"ID":"46543144-422d-406a-a81f-289c42535283","Type":"ContainerStarted","Data":"a4081c2d6b8c4159e69651a3d75b9ce945566fd2fbdf8cfad9702e40084b2321"}
Sep 19 11:35:20.961219 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:20.960723    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-w6hmz" podStartSLOduration=68.022181967 podStartE2EDuration="1m12.960706057s" totalImagesPullingTime="4.93852409s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 11:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:20.958859482 +0000 UTC m=+72.881172366" watchObservedRunningTime="2026-09-19 11:35:20.960706057 +0000 UTC m=+72.883018937"
Sep 19 11:35:22.966184 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:22.966152    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-lmpmn" event={"ID":"46543144-422d-406a-a81f-289c42535283","Type":"ContainerStarted","Data":"bbd953d649997739f2c78ac0a41b27158e289b0a0113996cfd759f3ea33d3785"}
Sep 19 11:35:22.966587 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:22.966380    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-lmpmn"
Sep 19 11:35:22.971175 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:22.971148    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-lmpmn"
Sep 19 11:35:22.981753 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:22.981709    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-lmpmn" podStartSLOduration=4.090501037 podStartE2EDuration="5.981694223s" totalImagesPullingTime="1.891193186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:35:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:22.980748098 +0000 UTC m=+74.903060979" watchObservedRunningTime="2026-09-19 11:35:22.981694223 +0000 UTC m=+74.904007105"
Sep 19 11:35:23.931367 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:23.927811    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"]
Sep 19 11:35:23.935339 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:23.935309    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"
Sep 19 11:35:23.938959 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:23.938794    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 19 11:35:23.939142 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:23.939120    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dfnlh\""
Sep 19 11:35:23.939252 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:23.939223    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 19 11:35:23.939312 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:23.939255    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 19 11:35:23.939463 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:23.939443    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 19 11:35:23.939762 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:23.939741    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 19 11:35:23.939996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:23.939973    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"]
Sep 19 11:35:24.021474 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.021438    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb5w8\" (UniqueName: \"kubernetes.io/projected/fea2f451-b87d-4f5d-a991-d3480ea0dc63-kube-api-access-lb5w8\") pod \"prometheus-operator-7bd5d56d6-97hh5\" (UID: \"fea2f451-b87d-4f5d-a991-d3480ea0dc63\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"
Sep 19 11:35:24.021936 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.021487    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fea2f451-b87d-4f5d-a991-d3480ea0dc63-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-97hh5\" (UID: \"fea2f451-b87d-4f5d-a991-d3480ea0dc63\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"
Sep 19 11:35:24.021936 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.021604    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fea2f451-b87d-4f5d-a991-d3480ea0dc63-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-97hh5\" (UID: \"fea2f451-b87d-4f5d-a991-d3480ea0dc63\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"
Sep 19 11:35:24.021936 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.021658    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fea2f451-b87d-4f5d-a991-d3480ea0dc63-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-97hh5\" (UID: \"fea2f451-b87d-4f5d-a991-d3480ea0dc63\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"
Sep 19 11:35:24.122816 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.122781    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb5w8\" (UniqueName: \"kubernetes.io/projected/fea2f451-b87d-4f5d-a991-d3480ea0dc63-kube-api-access-lb5w8\") pod \"prometheus-operator-7bd5d56d6-97hh5\" (UID: \"fea2f451-b87d-4f5d-a991-d3480ea0dc63\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"
Sep 19 11:35:24.122990 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.122826    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fea2f451-b87d-4f5d-a991-d3480ea0dc63-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-97hh5\" (UID: \"fea2f451-b87d-4f5d-a991-d3480ea0dc63\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"
Sep 19 11:35:24.122990 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:35:24.122937    2559 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 19 11:35:24.122990 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.122943    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fea2f451-b87d-4f5d-a991-d3480ea0dc63-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-97hh5\" (UID: \"fea2f451-b87d-4f5d-a991-d3480ea0dc63\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"
Sep 19 11:35:24.123162 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:35:24.123000    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea2f451-b87d-4f5d-a991-d3480ea0dc63-prometheus-operator-tls podName:fea2f451-b87d-4f5d-a991-d3480ea0dc63 nodeName:}" failed. No retries permitted until 2026-09-19 11:35:24.622980385 +0000 UTC m=+76.545293261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/fea2f451-b87d-4f5d-a991-d3480ea0dc63-prometheus-operator-tls") pod "prometheus-operator-7bd5d56d6-97hh5" (UID: "fea2f451-b87d-4f5d-a991-d3480ea0dc63") : secret "prometheus-operator-tls" not found
Sep 19 11:35:24.123162 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.123034    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fea2f451-b87d-4f5d-a991-d3480ea0dc63-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-97hh5\" (UID: \"fea2f451-b87d-4f5d-a991-d3480ea0dc63\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"
Sep 19 11:35:24.123786 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.123759    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fea2f451-b87d-4f5d-a991-d3480ea0dc63-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-97hh5\" (UID: \"fea2f451-b87d-4f5d-a991-d3480ea0dc63\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"
Sep 19 11:35:24.125803 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.125773    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fea2f451-b87d-4f5d-a991-d3480ea0dc63-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-97hh5\" (UID: \"fea2f451-b87d-4f5d-a991-d3480ea0dc63\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"
Sep 19 11:35:24.131178 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.131159    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb5w8\" (UniqueName: \"kubernetes.io/projected/fea2f451-b87d-4f5d-a991-d3480ea0dc63-kube-api-access-lb5w8\") pod \"prometheus-operator-7bd5d56d6-97hh5\" (UID: \"fea2f451-b87d-4f5d-a991-d3480ea0dc63\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"
Sep 19 11:35:24.628313 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.628270    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fea2f451-b87d-4f5d-a991-d3480ea0dc63-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-97hh5\" (UID: \"fea2f451-b87d-4f5d-a991-d3480ea0dc63\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"
Sep 19 11:35:24.631461 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.631433    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fea2f451-b87d-4f5d-a991-d3480ea0dc63-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-97hh5\" (UID: \"fea2f451-b87d-4f5d-a991-d3480ea0dc63\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"
Sep 19 11:35:24.846262 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.846232    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"
Sep 19 11:35:24.976286 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.975521    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b89cfd9b4-5lfjt" event={"ID":"cfbe13bc-b73b-4b57-bede-50e74ee50d57","Type":"ContainerStarted","Data":"abb41d001cc48b5590f1b054c10dea3c98a0012609e1684b55a8ce3ccf4dc8af"}
Sep 19 11:35:24.978206 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.978164    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9844b888d-plh7r" event={"ID":"70bed969-3852-4859-9d55-a1546255bf3e","Type":"ContainerStarted","Data":"f2a381e2b53c7ca0bb496d9b7be1359a24390a4e17e5de6ee48df510ee2f4bb3"}
Sep 19 11:35:24.992488 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:24.992436    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b89cfd9b4-5lfjt" podStartSLOduration=2.216821513 podStartE2EDuration="6.992417989s" totalImagesPullingTime="4.775596476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:35:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:24.991385678 +0000 UTC m=+76.913698557" watchObservedRunningTime="2026-09-19 11:35:24.992417989 +0000 UTC m=+76.914730870"
Sep 19 11:35:25.008178 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:25.008121    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9844b888d-plh7r" podStartSLOduration=1.217220297 podStartE2EDuration="6.0081006s" totalImagesPullingTime="4.790880303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:35:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:25.006918316 +0000 UTC m=+76.929231197" watchObservedRunningTime="2026-09-19 11:35:25.0081006 +0000 UTC m=+76.930413481"
Sep 19 11:35:25.142348 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:25.142283    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5"]
Sep 19 11:35:25.145040 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:35:25.145012    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea2f451_b87d_4f5d_a991_d3480ea0dc63.slice/crio-e233c03b089bb3500e0437d3370e7f5de4efa2250a70dd59160b94f99f121eda WatchSource:0}: Error finding container e233c03b089bb3500e0437d3370e7f5de4efa2250a70dd59160b94f99f121eda: Status 404 returned error can't find the container with id e233c03b089bb3500e0437d3370e7f5de4efa2250a70dd59160b94f99f121eda
Sep 19 11:35:25.983241 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:25.983204    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5" event={"ID":"fea2f451-b87d-4f5d-a991-d3480ea0dc63","Type":"ContainerStarted","Data":"e233c03b089bb3500e0437d3370e7f5de4efa2250a70dd59160b94f99f121eda"}
Sep 19 11:35:26.988253 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:26.988215    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5" event={"ID":"fea2f451-b87d-4f5d-a991-d3480ea0dc63","Type":"ContainerStarted","Data":"dd48f0de3a0ca7575952859a5ec9c67c7dd22048cbe524dd80ccb9528be1661b"}
Sep 19 11:35:26.988607 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:26.988269    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5" event={"ID":"fea2f451-b87d-4f5d-a991-d3480ea0dc63","Type":"ContainerStarted","Data":"0a23de133547ad8d015572c577cd3faea0d9cbcaf5bb3ad3d83699b04f1ebec7"}
Sep 19 11:35:27.004304 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:27.004257    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-97hh5" podStartSLOduration=2.727070251 podStartE2EDuration="4.004242933s" totalImagesPullingTime="1.277172682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:27.002782867 +0000 UTC m=+78.925095748" watchObservedRunningTime="2026-09-19 11:35:27.004242933 +0000 UTC m=+78.926555814"
Sep 19 11:35:28.453980 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:28.453944    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:28.454346 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:28.454005    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:28.458454 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:28.458432    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:28.997978 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:28.997953    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:29.261059 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.260986    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"]
Sep 19 11:35:29.263958 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.263943    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"
Sep 19 11:35:29.266396 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.266362    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 19 11:35:29.266513 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.266402    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-d45cf\""
Sep 19 11:35:29.266513 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.266446    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 19 11:35:29.277457 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.277431    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"]
Sep 19 11:35:29.286634 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.286612    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-sssb7"]
Sep 19 11:35:29.289553 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.289534    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.290631 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.290613    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-58dtg"]
Sep 19 11:35:29.293301 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.293275    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 19 11:35:29.293394 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.293315    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-mbsb4\""
Sep 19 11:35:29.293771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.293740    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 19 11:35:29.294128 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.294108    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 19 11:35:29.296760 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.296741    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.299501 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.299480    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 19 11:35:29.299608 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.299485    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 19 11:35:29.299719 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.299705    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5clwj\""
Sep 19 11:35:29.299771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.299709    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 19 11:35:29.300486 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.300463    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-sssb7"]
Sep 19 11:35:29.359672 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.359641    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/88bc1683-353e-4f02-ab36-8e5dce1dff76-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.359805 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.359680    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqsf2\" (UniqueName: \"kubernetes.io/projected/6efb3a91-0d2d-41f0-b169-9c6cda05cd2f-kube-api-access-nqsf2\") pod \"openshift-state-metrics-86c5bc79d6-v48hz\" (UID: \"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"
Sep 19 11:35:29.359805 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.359706    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-tls\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.359805 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.359726    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjvx8\" (UniqueName: \"kubernetes.io/projected/e2b33905-4886-44b2-82e5-e9b07c2a9b46-kube-api-access-bjvx8\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.359805 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.359791    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88bc1683-353e-4f02-ab36-8e5dce1dff76-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.359987 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.359846    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/88bc1683-353e-4f02-ab36-8e5dce1dff76-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.359987 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.359875    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6efb3a91-0d2d-41f0-b169-9c6cda05cd2f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-v48hz\" (UID: \"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"
Sep 19 11:35:29.359987 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.359907    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-textfile\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.359987 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.359949    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/88bc1683-353e-4f02-ab36-8e5dce1dff76-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.359987 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.359973    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-accelerators-collector-config\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.360240 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.359994    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6efb3a91-0d2d-41f0-b169-9c6cda05cd2f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-v48hz\" (UID: \"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"
Sep 19 11:35:29.360240 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.360011    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e2b33905-4886-44b2-82e5-e9b07c2a9b46-root\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.360240 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.360060    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx7wq\" (UniqueName: \"kubernetes.io/projected/88bc1683-353e-4f02-ab36-8e5dce1dff76-kube-api-access-xx7wq\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.360240 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.360110    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b33905-4886-44b2-82e5-e9b07c2a9b46-metrics-client-ca\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.360240 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.360139    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-wtmp\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.360240 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.360170    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6efb3a91-0d2d-41f0-b169-9c6cda05cd2f-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-v48hz\" (UID: \"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"
Sep 19 11:35:29.360240 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.360221    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2b33905-4886-44b2-82e5-e9b07c2a9b46-sys\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.360493 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.360245    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.360493 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.360296    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/88bc1683-353e-4f02-ab36-8e5dce1dff76-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.448507 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.448475    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:29.448651 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.448547    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:29.453026 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.453005    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:29.460827 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.460811    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6efb3a91-0d2d-41f0-b169-9c6cda05cd2f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-v48hz\" (UID: \"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"
Sep 19 11:35:29.461125 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.460832    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e2b33905-4886-44b2-82e5-e9b07c2a9b46-root\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.461125 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.460850    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx7wq\" (UniqueName: \"kubernetes.io/projected/88bc1683-353e-4f02-ab36-8e5dce1dff76-kube-api-access-xx7wq\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.461125 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.460866    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b33905-4886-44b2-82e5-e9b07c2a9b46-metrics-client-ca\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.461125 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.460881    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-wtmp\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.461125 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.460911    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6efb3a91-0d2d-41f0-b169-9c6cda05cd2f-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-v48hz\" (UID: \"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"
Sep 19 11:35:29.461125 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.460926    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e2b33905-4886-44b2-82e5-e9b07c2a9b46-root\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.461125 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.460934    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2b33905-4886-44b2-82e5-e9b07c2a9b46-sys\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.461125 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.460987    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.461125 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461014    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/88bc1683-353e-4f02-ab36-8e5dce1dff76-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.461125 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461045    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/88bc1683-353e-4f02-ab36-8e5dce1dff76-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.461125 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461056    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-wtmp\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.461125 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461080    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqsf2\" (UniqueName: \"kubernetes.io/projected/6efb3a91-0d2d-41f0-b169-9c6cda05cd2f-kube-api-access-nqsf2\") pod \"openshift-state-metrics-86c5bc79d6-v48hz\" (UID: \"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"
Sep 19 11:35:29.461719 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461353    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-tls\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.461719 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461382    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjvx8\" (UniqueName: \"kubernetes.io/projected/e2b33905-4886-44b2-82e5-e9b07c2a9b46-kube-api-access-bjvx8\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.461719 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461434    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88bc1683-353e-4f02-ab36-8e5dce1dff76-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.461719 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461478    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/88bc1683-353e-4f02-ab36-8e5dce1dff76-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.461719 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461488    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/88bc1683-353e-4f02-ab36-8e5dce1dff76-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.461719 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461505    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6efb3a91-0d2d-41f0-b169-9c6cda05cd2f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-v48hz\" (UID: \"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"
Sep 19 11:35:29.461719 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461539    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-textfile\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.461719 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461551    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b33905-4886-44b2-82e5-e9b07c2a9b46-metrics-client-ca\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.461719 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461574    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/88bc1683-353e-4f02-ab36-8e5dce1dff76-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.462170 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461768    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6efb3a91-0d2d-41f0-b169-9c6cda05cd2f-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-v48hz\" (UID: \"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"
Sep 19 11:35:29.462170 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461771    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-accelerators-collector-config\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.462170 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.461815    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2b33905-4886-44b2-82e5-e9b07c2a9b46-sys\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.462170 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:35:29.461898    2559 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 19 11:35:29.462170 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:35:29.461964    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-tls podName:e2b33905-4886-44b2-82e5-e9b07c2a9b46 nodeName:}" failed. No retries permitted until 2026-09-19 11:35:29.961944237 +0000 UTC m=+81.884257096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-tls") pod "node-exporter-58dtg" (UID: "e2b33905-4886-44b2-82e5-e9b07c2a9b46") : secret "node-exporter-tls" not found
Sep 19 11:35:29.462450 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.462248    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/88bc1683-353e-4f02-ab36-8e5dce1dff76-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.462450 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.462433    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-textfile\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.462558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.462449    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-accelerators-collector-config\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.462650 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.462622    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88bc1683-353e-4f02-ab36-8e5dce1dff76-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.463870 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.463845    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6efb3a91-0d2d-41f0-b169-9c6cda05cd2f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-v48hz\" (UID: \"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"
Sep 19 11:35:29.464083 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.464066    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/88bc1683-353e-4f02-ab36-8e5dce1dff76-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.464204 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.464167    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.464738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.464721    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6efb3a91-0d2d-41f0-b169-9c6cda05cd2f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-v48hz\" (UID: \"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"
Sep 19 11:35:29.464860 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.464842    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/88bc1683-353e-4f02-ab36-8e5dce1dff76-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.468496 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.468472    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx7wq\" (UniqueName: \"kubernetes.io/projected/88bc1683-353e-4f02-ab36-8e5dce1dff76-kube-api-access-xx7wq\") pod \"kube-state-metrics-84c885658c-sssb7\" (UID: \"88bc1683-353e-4f02-ab36-8e5dce1dff76\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.468705 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.468658    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqsf2\" (UniqueName: \"kubernetes.io/projected/6efb3a91-0d2d-41f0-b169-9c6cda05cd2f-kube-api-access-nqsf2\") pod \"openshift-state-metrics-86c5bc79d6-v48hz\" (UID: \"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"
Sep 19 11:35:29.474486 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.474461    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjvx8\" (UniqueName: \"kubernetes.io/projected/e2b33905-4886-44b2-82e5-e9b07c2a9b46-kube-api-access-bjvx8\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.573047 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.572983    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"
Sep 19 11:35:29.599961 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.599937    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7"
Sep 19 11:35:29.842826 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.842761    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz"]
Sep 19 11:35:29.845797 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:35:29.845772    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6efb3a91_0d2d_41f0_b169_9c6cda05cd2f.slice/crio-b39f718b131a1b14ecf8c4cb016100e5c9f56aedae1fa4a6b8137d6db7067e14 WatchSource:0}: Error finding container b39f718b131a1b14ecf8c4cb016100e5c9f56aedae1fa4a6b8137d6db7067e14: Status 404 returned error can't find the container with id b39f718b131a1b14ecf8c4cb016100e5c9f56aedae1fa4a6b8137d6db7067e14
Sep 19 11:35:29.874640 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.874609    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-sssb7"]
Sep 19 11:35:29.876813 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:35:29.876790    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88bc1683_353e_4f02_ab36_8e5dce1dff76.slice/crio-c5df51616caee7a2ce04b27a40d6fcbbd0cc590a6293262905010a9263642453 WatchSource:0}: Error finding container c5df51616caee7a2ce04b27a40d6fcbbd0cc590a6293262905010a9263642453: Status 404 returned error can't find the container with id c5df51616caee7a2ce04b27a40d6fcbbd0cc590a6293262905010a9263642453
Sep 19 11:35:29.964585 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.964558    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-tls\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.966631 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.966608    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e2b33905-4886-44b2-82e5-e9b07c2a9b46-node-exporter-tls\") pod \"node-exporter-58dtg\" (UID: \"e2b33905-4886-44b2-82e5-e9b07c2a9b46\") " pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:29.998797 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:29.998763    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7" event={"ID":"88bc1683-353e-4f02-ab36-8e5dce1dff76","Type":"ContainerStarted","Data":"c5df51616caee7a2ce04b27a40d6fcbbd0cc590a6293262905010a9263642453"}
Sep 19 11:35:30.000602 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.000578    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz" event={"ID":"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f","Type":"ContainerStarted","Data":"0b29423ec87842b1fe90d269e9a7ff76766d74884c63b95a2f880399cf9a2852"}
Sep 19 11:35:30.000696 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.000623    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz" event={"ID":"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f","Type":"ContainerStarted","Data":"0a7dbbef9c5719c8631fa50f01442678b17bbc99feef99720d7fb4a40ac44bd3"}
Sep 19 11:35:30.000696 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.000648    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz" event={"ID":"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f","Type":"ContainerStarted","Data":"b39f718b131a1b14ecf8c4cb016100e5c9f56aedae1fa4a6b8137d6db7067e14"}
Sep 19 11:35:30.004383 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.004360    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:35:30.051767 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.051688    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b89cfd9b4-5lfjt"]
Sep 19 11:35:30.207854 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.207819    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-58dtg"
Sep 19 11:35:30.218520 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:35:30.218472    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2b33905_4886_44b2_82e5_e9b07c2a9b46.slice/crio-b00a2e88d694ce5b4cb01cd16dd7ca1e65e5ec8b077fcce0af44f6343b67f67a WatchSource:0}: Error finding container b00a2e88d694ce5b4cb01cd16dd7ca1e65e5ec8b077fcce0af44f6343b67f67a: Status 404 returned error can't find the container with id b00a2e88d694ce5b4cb01cd16dd7ca1e65e5ec8b077fcce0af44f6343b67f67a
Sep 19 11:35:30.324694 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.324664    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 19 11:35:30.328461 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.328437    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.330868 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.330837    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 19 11:35:30.331035 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.330837    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 19 11:35:30.331170 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.331152    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 19 11:35:30.331170 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.331155    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 19 11:35:30.331170 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.331156    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 19 11:35:30.331170 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.331168    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 19 11:35:30.331510 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.331407    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 19 11:35:30.331510 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.331418    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 19 11:35:30.331510 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.331465    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 19 11:35:30.331663 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.331629    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kr57b\""
Sep 19 11:35:30.343747 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.343716    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 19 11:35:30.368871 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.368844    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdc7k\" (UniqueName: \"kubernetes.io/projected/fb1e3e58-c987-4673-8bb5-bfb72d011d62-kube-api-access-gdc7k\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.368968 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.368879    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb1e3e58-c987-4673-8bb5-bfb72d011d62-config-out\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.368968 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.368919    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.368968 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.368939    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.368968 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.368953    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb1e3e58-c987-4673-8bb5-bfb72d011d62-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.369161 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.368973    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.369161 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.368998    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-web-config\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.369161 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.369110    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.369161 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.369148    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb1e3e58-c987-4673-8bb5-bfb72d011d62-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.369380 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.369205    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1e3e58-c987-4673-8bb5-bfb72d011d62-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.369380 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.369248    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-config-volume\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.369380 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.369271    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.369380 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.369295    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb1e3e58-c987-4673-8bb5-bfb72d011d62-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.470056 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.469974    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdc7k\" (UniqueName: \"kubernetes.io/projected/fb1e3e58-c987-4673-8bb5-bfb72d011d62-kube-api-access-gdc7k\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.470056 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.470036    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb1e3e58-c987-4673-8bb5-bfb72d011d62-config-out\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.470558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.470084    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.470558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.470116    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.470558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.470142    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb1e3e58-c987-4673-8bb5-bfb72d011d62-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.470558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.470160    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.470558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.470176    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-web-config\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.470558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.470224    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.470558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.470240    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb1e3e58-c987-4673-8bb5-bfb72d011d62-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.470558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.470262    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1e3e58-c987-4673-8bb5-bfb72d011d62-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.470558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.470291    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-config-volume\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.470558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.470314    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.470558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.470335    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb1e3e58-c987-4673-8bb5-bfb72d011d62-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.471102 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.470645    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb1e3e58-c987-4673-8bb5-bfb72d011d62-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.471102 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:35:30.470735    2559 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 19 11:35:30.471102 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:35:30.470795    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-main-tls podName:fb1e3e58-c987-4673-8bb5-bfb72d011d62 nodeName:}" failed. No retries permitted until 2026-09-19 11:35:30.970775442 +0000 UTC m=+82.893088300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "fb1e3e58-c987-4673-8bb5-bfb72d011d62") : secret "alertmanager-main-tls" not found
Sep 19 11:35:30.471102 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:35:30.470954    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fb1e3e58-c987-4673-8bb5-bfb72d011d62-alertmanager-trusted-ca-bundle podName:fb1e3e58-c987-4673-8bb5-bfb72d011d62 nodeName:}" failed. No retries permitted until 2026-09-19 11:35:30.970930614 +0000 UTC m=+82.893243490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/fb1e3e58-c987-4673-8bb5-bfb72d011d62-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "fb1e3e58-c987-4673-8bb5-bfb72d011d62") : configmap references non-existent config key: ca-bundle.crt
Sep 19 11:35:30.472062 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.472002    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb1e3e58-c987-4673-8bb5-bfb72d011d62-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.474239 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.474156    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb1e3e58-c987-4673-8bb5-bfb72d011d62-config-out\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.474239 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.474159    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.474402 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.474331    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-config-volume\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.474402 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.474334    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-web-config\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.475492 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.475467    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb1e3e58-c987-4673-8bb5-bfb72d011d62-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.475601 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.475496    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.476735 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.476691    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.476872 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.476752    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.482056 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.482032    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdc7k\" (UniqueName: \"kubernetes.io/projected/fb1e3e58-c987-4673-8bb5-bfb72d011d62-kube-api-access-gdc7k\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.975758 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.975726    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.976052 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.975768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1e3e58-c987-4673-8bb5-bfb72d011d62-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.976622 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.976600    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1e3e58-c987-4673-8bb5-bfb72d011d62-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:30.978598 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:30.978578    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:31.006447 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:31.006412    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-58dtg" event={"ID":"e2b33905-4886-44b2-82e5-e9b07c2a9b46","Type":"ContainerStarted","Data":"b00a2e88d694ce5b4cb01cd16dd7ca1e65e5ec8b077fcce0af44f6343b67f67a"}
Sep 19 11:35:31.239580 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:31.239506    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:35:31.859326 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:31.859275    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 19 11:35:31.861429 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:35:31.861394    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb1e3e58_c987_4673_8bb5_bfb72d011d62.slice/crio-c5baf2b8899bf1be97ed5509bf0509200b08aa338ea85678b15823ef4c8f7561 WatchSource:0}: Error finding container c5baf2b8899bf1be97ed5509bf0509200b08aa338ea85678b15823ef4c8f7561: Status 404 returned error can't find the container with id c5baf2b8899bf1be97ed5509bf0509200b08aa338ea85678b15823ef4c8f7561
Sep 19 11:35:32.016552 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.016512    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb1e3e58-c987-4673-8bb5-bfb72d011d62","Type":"ContainerStarted","Data":"c5baf2b8899bf1be97ed5509bf0509200b08aa338ea85678b15823ef4c8f7561"}
Sep 19 11:35:32.018883 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.018852    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7" event={"ID":"88bc1683-353e-4f02-ab36-8e5dce1dff76","Type":"ContainerStarted","Data":"bad797b000e52339a75dd00e964c3cdc44b02e4855ae0d6bd841c58216a07119"}
Sep 19 11:35:32.019004 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.018910    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7" event={"ID":"88bc1683-353e-4f02-ab36-8e5dce1dff76","Type":"ContainerStarted","Data":"0e91ef643d55fcea2696dff4cc8609efd771484d58d555efc144d1934f4c508b"}
Sep 19 11:35:32.019004 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.018940    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7" event={"ID":"88bc1683-353e-4f02-ab36-8e5dce1dff76","Type":"ContainerStarted","Data":"18b7839f82acd8237cef41c3008992317e2eefd06ea6af380f128c87cb147980"}
Sep 19 11:35:32.020547 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.020518    2559 generic.go:356] "Generic (PLEG): container finished" podID="e2b33905-4886-44b2-82e5-e9b07c2a9b46" containerID="c8e845de5f0f60eb5613475f9a750efc5d5361f10e4884f6badd96993679c748" exitCode=0
Sep 19 11:35:32.020655 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.020603    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-58dtg" event={"ID":"e2b33905-4886-44b2-82e5-e9b07c2a9b46","Type":"ContainerDied","Data":"c8e845de5f0f60eb5613475f9a750efc5d5361f10e4884f6badd96993679c748"}
Sep 19 11:35:32.023853 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.023831    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz" event={"ID":"6efb3a91-0d2d-41f0-b169-9c6cda05cd2f","Type":"ContainerStarted","Data":"fbe71a8725aad4b18e3af2a41d1ea8a48279a5263dbd52e4b562c111b90ca23d"}
Sep 19 11:35:32.041412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.041367    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-sssb7" podStartSLOduration=1.343074354 podStartE2EDuration="3.041353553s" totalImagesPullingTime="1.698279199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:35:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:32.039552598 +0000 UTC m=+83.961865481" watchObservedRunningTime="2026-09-19 11:35:32.041353553 +0000 UTC m=+83.963666428"
Sep 19 11:35:32.071360 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.071313    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v48hz" podStartSLOduration=1.462004304 podStartE2EDuration="3.071294867s" totalImagesPullingTime="1.609290563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:35:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:32.070106269 +0000 UTC m=+83.992419152" watchObservedRunningTime="2026-09-19 11:35:32.071294867 +0000 UTC m=+83.993607749"
Sep 19 11:35:32.356475 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.356398    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"]
Sep 19 11:35:32.359957 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.359938    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.362620 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.362603    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8to48cb2mnga2\""
Sep 19 11:35:32.362620 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.362613    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 19 11:35:32.362800 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.362628    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 19 11:35:32.362800 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.362645    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 19 11:35:32.363085 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.363070    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 19 11:35:32.363137 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.363084    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 19 11:35:32.363203 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.363156    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-r28p8\""
Sep 19 11:35:32.370824 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.370802    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"]
Sep 19 11:35:32.489242 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.489212    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.489403 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.489253    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jldxk\" (UniqueName: \"kubernetes.io/projected/877d0665-b590-4c97-a37d-2c4672267afa-kube-api-access-jldxk\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.489403 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.489278    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-grpc-tls\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.489403 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.489302    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-thanos-querier-tls\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.489403 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.489342    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/877d0665-b590-4c97-a37d-2c4672267afa-metrics-client-ca\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.489631 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.489443    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.489631 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.489478    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.489631 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.489505    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.590292 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.590267    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.590424 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.590303    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.590424 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.590336    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.590424 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.590386    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.590424 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.590418    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jldxk\" (UniqueName: \"kubernetes.io/projected/877d0665-b590-4c97-a37d-2c4672267afa-kube-api-access-jldxk\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.590638 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.590445    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-grpc-tls\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.590638 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.590471    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-thanos-querier-tls\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.590638 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.590502    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/877d0665-b590-4c97-a37d-2c4672267afa-metrics-client-ca\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.591488 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.591433    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/877d0665-b590-4c97-a37d-2c4672267afa-metrics-client-ca\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.593505 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.593478    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.593718 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.593652    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.593822 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.593778    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-grpc-tls\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.594211 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.594163    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.597280 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.596075    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-thanos-querier-tls\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.597280 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.596897    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/877d0665-b590-4c97-a37d-2c4672267afa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.598393 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.598361    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jldxk\" (UniqueName: \"kubernetes.io/projected/877d0665-b590-4c97-a37d-2c4672267afa-kube-api-access-jldxk\") pod \"thanos-querier-7f94cf4d78-26l2m\" (UID: \"877d0665-b590-4c97-a37d-2c4672267afa\") " pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.669324 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.669249    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:32.948114 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:32.948088    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"]
Sep 19 11:35:32.971336 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:35:32.971313    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod877d0665_b590_4c97_a37d_2c4672267afa.slice/crio-fbad8bea791e311b3469f25788ee229d4f7a47d8d440406d342cfd14f6e95cf5 WatchSource:0}: Error finding container fbad8bea791e311b3469f25788ee229d4f7a47d8d440406d342cfd14f6e95cf5: Status 404 returned error can't find the container with id fbad8bea791e311b3469f25788ee229d4f7a47d8d440406d342cfd14f6e95cf5
Sep 19 11:35:33.030979 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.030951    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m" event={"ID":"877d0665-b590-4c97-a37d-2c4672267afa","Type":"ContainerStarted","Data":"fbad8bea791e311b3469f25788ee229d4f7a47d8d440406d342cfd14f6e95cf5"}
Sep 19 11:35:33.033402 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.033377    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-58dtg" event={"ID":"e2b33905-4886-44b2-82e5-e9b07c2a9b46","Type":"ContainerStarted","Data":"a79bedc91df25bebeff12c751d46f5107d2882baccae529871502f44bf50b4ab"}
Sep 19 11:35:33.033497 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.033437    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-58dtg" event={"ID":"e2b33905-4886-44b2-82e5-e9b07c2a9b46","Type":"ContainerStarted","Data":"7ab0665df16ba03b76041a62768f12df9793419e6ab33ef43f72f97e62f75c93"}
Sep 19 11:35:33.055267 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.055223    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-58dtg" podStartSLOduration=2.609238713 podStartE2EDuration="4.055208353s" totalImagesPullingTime="1.356399107s" totalInitContainerRuntime="89.570533ms" isStatefulPod=true podCreationTimestamp="2026-09-19 11:35:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:33.050043748 +0000 UTC m=+84.972356687" watchObservedRunningTime="2026-09-19 11:35:33.055208353 +0000 UTC m=+84.977521234"
Sep 19 11:35:33.564163 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.564128    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6f59cf996d-74dtw"]
Sep 19 11:35:33.568079 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.568056    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.571375 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.571312    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-eeek0j54jj7jd\""
Sep 19 11:35:33.571375 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.571338    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 19 11:35:33.571375 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.571352    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 19 11:35:33.571662 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.571638    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-hmjlw\""
Sep 19 11:35:33.571781 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.571706    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 19 11:35:33.571781 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.571638    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 19 11:35:33.576356 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.576335    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f59cf996d-74dtw"]
Sep 19 11:35:33.701688 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.701658    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/204107b3-623b-4315-a136-1e895bed9f1d-audit-log\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.701688 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.701689    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/204107b3-623b-4315-a136-1e895bed9f1d-client-ca-bundle\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.701962 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.701723    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/204107b3-623b-4315-a136-1e895bed9f1d-secret-metrics-server-client-certs\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.701962 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.701753    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/204107b3-623b-4315-a136-1e895bed9f1d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.701962 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.701854    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/204107b3-623b-4315-a136-1e895bed9f1d-secret-metrics-server-tls\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.701962 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.701952    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/204107b3-623b-4315-a136-1e895bed9f1d-metrics-server-audit-profiles\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.702222 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.701981    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt6vs\" (UniqueName: \"kubernetes.io/projected/204107b3-623b-4315-a136-1e895bed9f1d-kube-api-access-mt6vs\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.802974 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.802935    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/204107b3-623b-4315-a136-1e895bed9f1d-metrics-server-audit-profiles\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.803132 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.802981    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt6vs\" (UniqueName: \"kubernetes.io/projected/204107b3-623b-4315-a136-1e895bed9f1d-kube-api-access-mt6vs\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.803132 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.803099    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/204107b3-623b-4315-a136-1e895bed9f1d-audit-log\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.803249 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.803132    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/204107b3-623b-4315-a136-1e895bed9f1d-client-ca-bundle\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.803249 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.803162    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/204107b3-623b-4315-a136-1e895bed9f1d-secret-metrics-server-client-certs\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.803352 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.803252    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/204107b3-623b-4315-a136-1e895bed9f1d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.803352 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.803304    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/204107b3-623b-4315-a136-1e895bed9f1d-secret-metrics-server-tls\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.803462 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.803444    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/204107b3-623b-4315-a136-1e895bed9f1d-audit-log\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.803928 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.803847    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/204107b3-623b-4315-a136-1e895bed9f1d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.804160 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.804024    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/204107b3-623b-4315-a136-1e895bed9f1d-metrics-server-audit-profiles\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.805763 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.805733    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/204107b3-623b-4315-a136-1e895bed9f1d-secret-metrics-server-client-certs\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.805877 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.805784    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/204107b3-623b-4315-a136-1e895bed9f1d-secret-metrics-server-tls\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.805945 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.805913    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/204107b3-623b-4315-a136-1e895bed9f1d-client-ca-bundle\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.810872 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.810853    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt6vs\" (UniqueName: \"kubernetes.io/projected/204107b3-623b-4315-a136-1e895bed9f1d-kube-api-access-mt6vs\") pod \"metrics-server-6f59cf996d-74dtw\" (UID: \"204107b3-623b-4315-a136-1e895bed9f1d\") " pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:33.878057 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:33.877899    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:34.041053 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.041019    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerID="03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead" exitCode=0
Sep 19 11:35:34.042081 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.041284    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb1e3e58-c987-4673-8bb5-bfb72d011d62","Type":"ContainerDied","Data":"03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead"}
Sep 19 11:35:34.160571 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.160542    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f59cf996d-74dtw"]
Sep 19 11:35:34.162781 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:35:34.162758    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod204107b3_623b_4315_a136_1e895bed9f1d.slice/crio-f954422d19f6fee5bc328770b8e79fc7fbe090ca0c6d966adf93191c5ea6bc85 WatchSource:0}: Error finding container f954422d19f6fee5bc328770b8e79fc7fbe090ca0c6d966adf93191c5ea6bc85: Status 404 returned error can't find the container with id f954422d19f6fee5bc328770b8e79fc7fbe090ca0c6d966adf93191c5ea6bc85
Sep 19 11:35:34.467886 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.467854    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"]
Sep 19 11:35:34.472404 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.472386    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.475223 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.475089    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 19 11:35:34.475223 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.475101    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 19 11:35:34.475223 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.475162    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 19 11:35:34.475223 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.475203    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 19 11:35:34.475223 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.475205    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-9f799\""
Sep 19 11:35:34.475453 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.475206    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 19 11:35:34.480084 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.480067    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 19 11:35:34.484036 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.484016    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"]
Sep 19 11:35:34.611299 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.611266    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2b9901c8-0abe-466e-980d-67aafdfe730f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.611460 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.611323    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2b9901c8-0abe-466e-980d-67aafdfe730f-secret-telemeter-client\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.611460 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.611372    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b9901c8-0abe-466e-980d-67aafdfe730f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.611460 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.611454    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b9901c8-0abe-466e-980d-67aafdfe730f-serving-certs-ca-bundle\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.611620 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.611518    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx898\" (UniqueName: \"kubernetes.io/projected/2b9901c8-0abe-466e-980d-67aafdfe730f-kube-api-access-gx898\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.611620 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.611564    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2b9901c8-0abe-466e-980d-67aafdfe730f-telemeter-client-tls\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.611620 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.611608    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b9901c8-0abe-466e-980d-67aafdfe730f-metrics-client-ca\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.611764 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.611649    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2b9901c8-0abe-466e-980d-67aafdfe730f-federate-client-tls\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.712744 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.712700    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2b9901c8-0abe-466e-980d-67aafdfe730f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.712901 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.712756    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2b9901c8-0abe-466e-980d-67aafdfe730f-secret-telemeter-client\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.712901 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.712798    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b9901c8-0abe-466e-980d-67aafdfe730f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.712901 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.712824    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b9901c8-0abe-466e-980d-67aafdfe730f-serving-certs-ca-bundle\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.712901 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.712875    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx898\" (UniqueName: \"kubernetes.io/projected/2b9901c8-0abe-466e-980d-67aafdfe730f-kube-api-access-gx898\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.713120 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.712906    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2b9901c8-0abe-466e-980d-67aafdfe730f-telemeter-client-tls\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.713120 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.712930    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b9901c8-0abe-466e-980d-67aafdfe730f-metrics-client-ca\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.713120 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.712960    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2b9901c8-0abe-466e-980d-67aafdfe730f-federate-client-tls\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.714111 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.714065    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b9901c8-0abe-466e-980d-67aafdfe730f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.714111 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.714088    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b9901c8-0abe-466e-980d-67aafdfe730f-metrics-client-ca\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.714325 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.714250    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b9901c8-0abe-466e-980d-67aafdfe730f-serving-certs-ca-bundle\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.715674 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.715651    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2b9901c8-0abe-466e-980d-67aafdfe730f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.717854 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.717803    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2b9901c8-0abe-466e-980d-67aafdfe730f-secret-telemeter-client\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.718052 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.718004    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2b9901c8-0abe-466e-980d-67aafdfe730f-federate-client-tls\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.719350 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.719325    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2b9901c8-0abe-466e-980d-67aafdfe730f-telemeter-client-tls\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.721357 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.721335    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx898\" (UniqueName: \"kubernetes.io/projected/2b9901c8-0abe-466e-980d-67aafdfe730f-kube-api-access-gx898\") pod \"telemeter-client-64d7fb9d6d-8pqcs\" (UID: \"2b9901c8-0abe-466e-980d-67aafdfe730f\") " pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:34.782307 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:34.782260    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"
Sep 19 11:35:35.048535 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:35.048441    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw" event={"ID":"204107b3-623b-4315-a136-1e895bed9f1d","Type":"ContainerStarted","Data":"f954422d19f6fee5bc328770b8e79fc7fbe090ca0c6d966adf93191c5ea6bc85"}
Sep 19 11:35:36.715621 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:36.715596    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs"]
Sep 19 11:35:36.717645 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:35:36.717561    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b9901c8_0abe_466e_980d_67aafdfe730f.slice/crio-8fb50f26bdd6d0c6f7bb7c23c0ee288bda41c26396a9f92a2752ae552f84c6dd WatchSource:0}: Error finding container 8fb50f26bdd6d0c6f7bb7c23c0ee288bda41c26396a9f92a2752ae552f84c6dd: Status 404 returned error can't find the container with id 8fb50f26bdd6d0c6f7bb7c23c0ee288bda41c26396a9f92a2752ae552f84c6dd
Sep 19 11:35:37.057635 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:37.057590    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs" event={"ID":"2b9901c8-0abe-466e-980d-67aafdfe730f","Type":"ContainerStarted","Data":"8fb50f26bdd6d0c6f7bb7c23c0ee288bda41c26396a9f92a2752ae552f84c6dd"}
Sep 19 11:35:37.059388 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:37.059355    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw" event={"ID":"204107b3-623b-4315-a136-1e895bed9f1d","Type":"ContainerStarted","Data":"f01f7f824376d76a8da6f0ec15b3aacca558156f8381bf407abffbc72796e0ad"}
Sep 19 11:35:37.062889 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:37.062865    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb1e3e58-c987-4673-8bb5-bfb72d011d62","Type":"ContainerStarted","Data":"b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81"}
Sep 19 11:35:37.063020 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:37.062933    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb1e3e58-c987-4673-8bb5-bfb72d011d62","Type":"ContainerStarted","Data":"c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661"}
Sep 19 11:35:37.063020 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:37.062977    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb1e3e58-c987-4673-8bb5-bfb72d011d62","Type":"ContainerStarted","Data":"ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6"}
Sep 19 11:35:37.063020 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:37.063014    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb1e3e58-c987-4673-8bb5-bfb72d011d62","Type":"ContainerStarted","Data":"4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820"}
Sep 19 11:35:37.063154 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:37.063050    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb1e3e58-c987-4673-8bb5-bfb72d011d62","Type":"ContainerStarted","Data":"0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a"}
Sep 19 11:35:37.065173 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:37.065151    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m" event={"ID":"877d0665-b590-4c97-a37d-2c4672267afa","Type":"ContainerStarted","Data":"987a3a5f9b61de73afc8b33f0d896e0e1eb7fc813f9921b312091f45836d5a79"}
Sep 19 11:35:37.065278 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:37.065227    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m" event={"ID":"877d0665-b590-4c97-a37d-2c4672267afa","Type":"ContainerStarted","Data":"2dd4f5947c4c543c403c74b1c483a53ed4f093960de9c260da2a45cb8a2e0718"}
Sep 19 11:35:37.065278 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:37.065252    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m" event={"ID":"877d0665-b590-4c97-a37d-2c4672267afa","Type":"ContainerStarted","Data":"0a1292166631df8da09b99186f1ee16be22b67a57b12b78effdcf0b32b528962"}
Sep 19 11:35:37.075925 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:37.075886    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw" podStartSLOduration=1.791789513 podStartE2EDuration="4.075873716s" totalImagesPullingTime="2.284084203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:35:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:37.074116553 +0000 UTC m=+88.996429428" watchObservedRunningTime="2026-09-19 11:35:37.075873716 +0000 UTC m=+88.998186597"
Sep 19 11:35:38.074217 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:38.074152    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb1e3e58-c987-4673-8bb5-bfb72d011d62","Type":"ContainerStarted","Data":"00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f"}
Sep 19 11:35:38.077368 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:38.077333    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m" event={"ID":"877d0665-b590-4c97-a37d-2c4672267afa","Type":"ContainerStarted","Data":"2235fa95e8e0b6278992c4b2bd792e7d30dff66aa75384e929d803a3bf02755c"}
Sep 19 11:35:38.077514 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:38.077401    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m" event={"ID":"877d0665-b590-4c97-a37d-2c4672267afa","Type":"ContainerStarted","Data":"a1eff32d159dbf48fbd74894a0dc98a4e6de74a303901115f0a519e0b28bf479"}
Sep 19 11:35:38.077514 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:38.077442    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m" event={"ID":"877d0665-b590-4c97-a37d-2c4672267afa","Type":"ContainerStarted","Data":"4139d0f47d82d6c4a421fcf8efe21d2fbefe213f27320078cfd3c3a5a1f48d34"}
Sep 19 11:35:38.098976 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:38.098917    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.687392042 podStartE2EDuration="8.098894294s" totalImagesPullingTime="4.391164406s" totalInitContainerRuntime="20.337846ms" isStatefulPod=false podCreationTimestamp="2026-09-19 11:35:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:38.096896887 +0000 UTC m=+90.019209807" watchObservedRunningTime="2026-09-19 11:35:38.098894294 +0000 UTC m=+90.021207209"
Sep 19 11:35:38.117257 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:38.117184    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m" podStartSLOduration=1.627899199 podStartE2EDuration="6.117165593s" totalImagesPullingTime="4.489266394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:35:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:38.116425608 +0000 UTC m=+90.038738485" watchObservedRunningTime="2026-09-19 11:35:38.117165593 +0000 UTC m=+90.039478475"
Sep 19 11:35:39.083252 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.083217    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs" event={"ID":"2b9901c8-0abe-466e-980d-67aafdfe730f","Type":"ContainerStarted","Data":"01982066f82f4cba2d5e7199c229728d5d8fd4f9be3744e1a3c714f3f16da251"}
Sep 19 11:35:39.083676 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.083317    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs" event={"ID":"2b9901c8-0abe-466e-980d-67aafdfe730f","Type":"ContainerStarted","Data":"d2842257108cff708a421dae85f9203af6ac410bc17365bf634c26470581ef2b"}
Sep 19 11:35:39.083676 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.083350    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs" event={"ID":"2b9901c8-0abe-466e-980d-67aafdfe730f","Type":"ContainerStarted","Data":"cd8bed7bae80ec553aeb4fa644d69b0a8e90a8c28d7bd3ecaebdbf3943689bab"}
Sep 19 11:35:39.083964 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.083943    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:39.102565 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.102527    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-64d7fb9d6d-8pqcs" podStartSLOduration=3.267526734 podStartE2EDuration="5.102514693s" totalImagesPullingTime="1.834987959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:39.101531688 +0000 UTC m=+91.023844569" watchObservedRunningTime="2026-09-19 11:35:39.102514693 +0000 UTC m=+91.024827575"
Sep 19 11:35:39.220080 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.220050    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dfc6fbdf4-ffs6p"]
Sep 19 11:35:39.223466 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.223451    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.231638 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.231614    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dfc6fbdf4-ffs6p"]
Sep 19 11:35:39.358322 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.358247    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0f82a6a-fd78-495f-be13-5626f133e611-console-serving-cert\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.358322 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.358280    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxgzb\" (UniqueName: \"kubernetes.io/projected/c0f82a6a-fd78-495f-be13-5626f133e611-kube-api-access-jxgzb\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.358322 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.358307    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-service-ca\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.358531 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.358372    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-console-config\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.358531 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.358412    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0f82a6a-fd78-495f-be13-5626f133e611-console-oauth-config\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.358531 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.358441    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-trusted-ca-bundle\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.358531 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.358457    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-oauth-serving-cert\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.459540 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.459506    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0f82a6a-fd78-495f-be13-5626f133e611-console-oauth-config\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.459661 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.459570    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-trusted-ca-bundle\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.459661 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.459594    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-oauth-serving-cert\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.459661 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.459639    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0f82a6a-fd78-495f-be13-5626f133e611-console-serving-cert\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.459825 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.459662    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxgzb\" (UniqueName: \"kubernetes.io/projected/c0f82a6a-fd78-495f-be13-5626f133e611-kube-api-access-jxgzb\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.459825 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.459703    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-service-ca\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.459825 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.459746    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-console-config\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.460567 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.460537    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-console-config\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.460704 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.460548    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-trusted-ca-bundle\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.460704 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.460657    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-oauth-serving-cert\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.460704 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.460661    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-service-ca\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.462506 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.462478    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0f82a6a-fd78-495f-be13-5626f133e611-console-oauth-config\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.462869 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.462847    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0f82a6a-fd78-495f-be13-5626f133e611-console-serving-cert\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.467817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.467795    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxgzb\" (UniqueName: \"kubernetes.io/projected/c0f82a6a-fd78-495f-be13-5626f133e611-kube-api-access-jxgzb\") pod \"console-6dfc6fbdf4-ffs6p\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") " pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.532823 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.532789    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:39.808762 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:39.808726    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dfc6fbdf4-ffs6p"]
Sep 19 11:35:39.812738 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:35:39.812707    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f82a6a_fd78_495f_be13_5626f133e611.slice/crio-3e3c8bbb0bf729b04767cced37e054366ad20d1ffc01976dcb4f48098e5be74e WatchSource:0}: Error finding container 3e3c8bbb0bf729b04767cced37e054366ad20d1ffc01976dcb4f48098e5be74e: Status 404 returned error can't find the container with id 3e3c8bbb0bf729b04767cced37e054366ad20d1ffc01976dcb4f48098e5be74e
Sep 19 11:35:40.089442 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:40.089350    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfc6fbdf4-ffs6p" event={"ID":"c0f82a6a-fd78-495f-be13-5626f133e611","Type":"ContainerStarted","Data":"f4b1aa62965734c610da617a98862e68ec1d7c5e75b8ff1aef65e04f50eafd4e"}
Sep 19 11:35:40.089442 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:40.089410    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfc6fbdf4-ffs6p" event={"ID":"c0f82a6a-fd78-495f-be13-5626f133e611","Type":"ContainerStarted","Data":"3e3c8bbb0bf729b04767cced37e054366ad20d1ffc01976dcb4f48098e5be74e"}
Sep 19 11:35:40.104936 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:40.104890    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dfc6fbdf4-ffs6p" podStartSLOduration=1.104874687 podStartE2EDuration="1.104874687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:35:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:35:40.103411708 +0000 UTC m=+92.025724588" watchObservedRunningTime="2026-09-19 11:35:40.104874687 +0000 UTC m=+92.027187568"
Sep 19 11:35:45.095806 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:45.095776    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7f94cf4d78-26l2m"
Sep 19 11:35:48.603557 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:48.603496    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pz68l_3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3/serve-healthcheck-canary/0.log"
Sep 19 11:35:49.533896 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:49.533850    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:49.534092 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:49.533922    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:49.538062 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:49.538039    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:50.137247 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:50.137209    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:35:50.179994 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:50.179967    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9844b888d-plh7r"]
Sep 19 11:35:50.959536 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:50.959510    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lslq5"
Sep 19 11:35:53.878272 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:53.878238    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:53.878632 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:53.878294    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:35:56.027078 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.027011    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b89cfd9b4-5lfjt" podUID="cfbe13bc-b73b-4b57-bede-50e74ee50d57" containerName="console" containerID="cri-o://abb41d001cc48b5590f1b054c10dea3c98a0012609e1684b55a8ce3ccf4dc8af" gracePeriod=15
Sep 19 11:35:56.160630 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.160595    2559 generic.go:356] "Generic (PLEG): container finished" podID="cfbe13bc-b73b-4b57-bede-50e74ee50d57" containerID="abb41d001cc48b5590f1b054c10dea3c98a0012609e1684b55a8ce3ccf4dc8af" exitCode=0
Sep 19 11:35:56.160756 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.160665    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b89cfd9b4-5lfjt" event={"ID":"cfbe13bc-b73b-4b57-bede-50e74ee50d57","Type":"ContainerDied","Data":"abb41d001cc48b5590f1b054c10dea3c98a0012609e1684b55a8ce3ccf4dc8af"}
Sep 19 11:35:56.325981 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.325957    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:56.505355 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.505324    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-oauth-config\") pod \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") "
Sep 19 11:35:56.505355 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.505361    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-serving-cert\") pod \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") "
Sep 19 11:35:56.505590 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.505393    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-config\") pod \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") "
Sep 19 11:35:56.505590 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.505412    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-service-ca\") pod \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") "
Sep 19 11:35:56.505590 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.505481    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6mjb\" (UniqueName: \"kubernetes.io/projected/cfbe13bc-b73b-4b57-bede-50e74ee50d57-kube-api-access-f6mjb\") pod \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") "
Sep 19 11:35:56.505590 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.505500    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-oauth-serving-cert\") pod \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\" (UID: \"cfbe13bc-b73b-4b57-bede-50e74ee50d57\") "
Sep 19 11:35:56.505860 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.505832    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-config" (OuterVolumeSpecName: "console-config") pod "cfbe13bc-b73b-4b57-bede-50e74ee50d57" (UID: "cfbe13bc-b73b-4b57-bede-50e74ee50d57"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:35:56.505973 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.505874    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-service-ca" (OuterVolumeSpecName: "service-ca") pod "cfbe13bc-b73b-4b57-bede-50e74ee50d57" (UID: "cfbe13bc-b73b-4b57-bede-50e74ee50d57"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:35:56.506019 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.505958    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cfbe13bc-b73b-4b57-bede-50e74ee50d57" (UID: "cfbe13bc-b73b-4b57-bede-50e74ee50d57"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:35:56.507860 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.507835    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cfbe13bc-b73b-4b57-bede-50e74ee50d57" (UID: "cfbe13bc-b73b-4b57-bede-50e74ee50d57"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:35:56.507996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.507977    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cfbe13bc-b73b-4b57-bede-50e74ee50d57" (UID: "cfbe13bc-b73b-4b57-bede-50e74ee50d57"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:35:56.508049 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.507981    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfbe13bc-b73b-4b57-bede-50e74ee50d57-kube-api-access-f6mjb" (OuterVolumeSpecName: "kube-api-access-f6mjb") pod "cfbe13bc-b73b-4b57-bede-50e74ee50d57" (UID: "cfbe13bc-b73b-4b57-bede-50e74ee50d57"). InnerVolumeSpecName "kube-api-access-f6mjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:35:56.606304 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.606284    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6mjb\" (UniqueName: \"kubernetes.io/projected/cfbe13bc-b73b-4b57-bede-50e74ee50d57-kube-api-access-f6mjb\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:35:56.606304 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.606305    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-oauth-serving-cert\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:35:56.606426 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.606314    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-oauth-config\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:35:56.606426 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.606322    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-serving-cert\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:35:56.606426 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.606331    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-console-config\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:35:56.606426 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:56.606339    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cfbe13bc-b73b-4b57-bede-50e74ee50d57-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:35:57.165970 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:57.165941    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b89cfd9b4-5lfjt"
Sep 19 11:35:57.165970 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:57.165965    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b89cfd9b4-5lfjt" event={"ID":"cfbe13bc-b73b-4b57-bede-50e74ee50d57","Type":"ContainerDied","Data":"e8f2c7a88eab11de1cbce110c2ce242369d58dd4b725db214065e3580f947e83"}
Sep 19 11:35:57.166506 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:57.166021    2559 scope.go:117] "RemoveContainer" containerID="abb41d001cc48b5590f1b054c10dea3c98a0012609e1684b55a8ce3ccf4dc8af"
Sep 19 11:35:57.180704 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:57.180683    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b89cfd9b4-5lfjt"]
Sep 19 11:35:57.183887 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:57.183867    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b89cfd9b4-5lfjt"]
Sep 19 11:35:58.593535 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:35:58.593503    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfbe13bc-b73b-4b57-bede-50e74ee50d57" path="/var/lib/kubelet/pods/cfbe13bc-b73b-4b57-bede-50e74ee50d57/volumes"
Sep 19 11:36:13.883328 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:13.883298    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:36:13.887038 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:13.887015    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6f59cf996d-74dtw"
Sep 19 11:36:15.199895 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.199856    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9844b888d-plh7r" podUID="70bed969-3852-4859-9d55-a1546255bf3e" containerName="console" containerID="cri-o://f2a381e2b53c7ca0bb496d9b7be1359a24390a4e17e5de6ee48df510ee2f4bb3" gracePeriod=15
Sep 19 11:36:15.439567 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.439544    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:36:15.564665 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.564570    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rfjk\" (UniqueName: \"kubernetes.io/projected/70bed969-3852-4859-9d55-a1546255bf3e-kube-api-access-9rfjk\") pod \"70bed969-3852-4859-9d55-a1546255bf3e\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") "
Sep 19 11:36:15.564665 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.564614    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-trusted-ca-bundle\") pod \"70bed969-3852-4859-9d55-a1546255bf3e\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") "
Sep 19 11:36:15.564665 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.564633    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70bed969-3852-4859-9d55-a1546255bf3e-console-oauth-config\") pod \"70bed969-3852-4859-9d55-a1546255bf3e\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") "
Sep 19 11:36:15.564942 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.564702    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-service-ca\") pod \"70bed969-3852-4859-9d55-a1546255bf3e\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") "
Sep 19 11:36:15.564942 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.564724    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-console-config\") pod \"70bed969-3852-4859-9d55-a1546255bf3e\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") "
Sep 19 11:36:15.564942 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.564760    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70bed969-3852-4859-9d55-a1546255bf3e-console-serving-cert\") pod \"70bed969-3852-4859-9d55-a1546255bf3e\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") "
Sep 19 11:36:15.564942 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.564784    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-oauth-serving-cert\") pod \"70bed969-3852-4859-9d55-a1546255bf3e\" (UID: \"70bed969-3852-4859-9d55-a1546255bf3e\") "
Sep 19 11:36:15.565413 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.565294    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "70bed969-3852-4859-9d55-a1546255bf3e" (UID: "70bed969-3852-4859-9d55-a1546255bf3e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:36:15.565413 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.565303    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-console-config" (OuterVolumeSpecName: "console-config") pod "70bed969-3852-4859-9d55-a1546255bf3e" (UID: "70bed969-3852-4859-9d55-a1546255bf3e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:36:15.565627 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.565564    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-service-ca" (OuterVolumeSpecName: "service-ca") pod "70bed969-3852-4859-9d55-a1546255bf3e" (UID: "70bed969-3852-4859-9d55-a1546255bf3e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:36:15.565627 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.565612    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "70bed969-3852-4859-9d55-a1546255bf3e" (UID: "70bed969-3852-4859-9d55-a1546255bf3e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:36:15.566992 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.566972    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70bed969-3852-4859-9d55-a1546255bf3e-kube-api-access-9rfjk" (OuterVolumeSpecName: "kube-api-access-9rfjk") pod "70bed969-3852-4859-9d55-a1546255bf3e" (UID: "70bed969-3852-4859-9d55-a1546255bf3e"). InnerVolumeSpecName "kube-api-access-9rfjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:36:15.567753 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.567723    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70bed969-3852-4859-9d55-a1546255bf3e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "70bed969-3852-4859-9d55-a1546255bf3e" (UID: "70bed969-3852-4859-9d55-a1546255bf3e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:36:15.567872 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.567784    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70bed969-3852-4859-9d55-a1546255bf3e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "70bed969-3852-4859-9d55-a1546255bf3e" (UID: "70bed969-3852-4859-9d55-a1546255bf3e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:36:15.666255 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.666217    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:15.666255 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.666252    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-console-config\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:15.666255 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.666261    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70bed969-3852-4859-9d55-a1546255bf3e-console-serving-cert\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:15.666255 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.666270    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-oauth-serving-cert\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:15.666480 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.666278    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rfjk\" (UniqueName: \"kubernetes.io/projected/70bed969-3852-4859-9d55-a1546255bf3e-kube-api-access-9rfjk\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:15.666480 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.666287    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70bed969-3852-4859-9d55-a1546255bf3e-trusted-ca-bundle\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:15.666480 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:15.666295    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70bed969-3852-4859-9d55-a1546255bf3e-console-oauth-config\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:16.250934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:16.250895    2559 generic.go:356] "Generic (PLEG): container finished" podID="70bed969-3852-4859-9d55-a1546255bf3e" containerID="f2a381e2b53c7ca0bb496d9b7be1359a24390a4e17e5de6ee48df510ee2f4bb3" exitCode=0
Sep 19 11:36:16.251447 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:16.250960    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9844b888d-plh7r"
Sep 19 11:36:16.251447 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:16.250977    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9844b888d-plh7r" event={"ID":"70bed969-3852-4859-9d55-a1546255bf3e","Type":"ContainerDied","Data":"f2a381e2b53c7ca0bb496d9b7be1359a24390a4e17e5de6ee48df510ee2f4bb3"}
Sep 19 11:36:16.251447 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:16.251030    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9844b888d-plh7r" event={"ID":"70bed969-3852-4859-9d55-a1546255bf3e","Type":"ContainerDied","Data":"5ff0127f2c21a82a3480d89394ed131c3e971906104041fe992cae80852cfc63"}
Sep 19 11:36:16.251447 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:16.251050    2559 scope.go:117] "RemoveContainer" containerID="f2a381e2b53c7ca0bb496d9b7be1359a24390a4e17e5de6ee48df510ee2f4bb3"
Sep 19 11:36:16.260466 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:16.260452    2559 scope.go:117] "RemoveContainer" containerID="f2a381e2b53c7ca0bb496d9b7be1359a24390a4e17e5de6ee48df510ee2f4bb3"
Sep 19 11:36:16.260749 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:36:16.260729    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2a381e2b53c7ca0bb496d9b7be1359a24390a4e17e5de6ee48df510ee2f4bb3\": container with ID starting with f2a381e2b53c7ca0bb496d9b7be1359a24390a4e17e5de6ee48df510ee2f4bb3 not found: ID does not exist" containerID="f2a381e2b53c7ca0bb496d9b7be1359a24390a4e17e5de6ee48df510ee2f4bb3"
Sep 19 11:36:16.260815 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:16.260755    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a381e2b53c7ca0bb496d9b7be1359a24390a4e17e5de6ee48df510ee2f4bb3"} err="failed to get container status \"f2a381e2b53c7ca0bb496d9b7be1359a24390a4e17e5de6ee48df510ee2f4bb3\": rpc error: code = NotFound desc = could not find container \"f2a381e2b53c7ca0bb496d9b7be1359a24390a4e17e5de6ee48df510ee2f4bb3\": container with ID starting with f2a381e2b53c7ca0bb496d9b7be1359a24390a4e17e5de6ee48df510ee2f4bb3 not found: ID does not exist"
Sep 19 11:36:16.270834 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:16.270812    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9844b888d-plh7r"]
Sep 19 11:36:16.274571 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:16.274552    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9844b888d-plh7r"]
Sep 19 11:36:16.594171 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:16.594094    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70bed969-3852-4859-9d55-a1546255bf3e" path="/var/lib/kubelet/pods/70bed969-3852-4859-9d55-a1546255bf3e/volumes"
Sep 19 11:36:49.565980 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:49.565944    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 19 11:36:49.566515 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:49.566292    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="alertmanager" containerID="cri-o://0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a" gracePeriod=120
Sep 19 11:36:49.566515 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:49.566304    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="kube-rbac-proxy" containerID="cri-o://c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661" gracePeriod=120
Sep 19 11:36:49.566515 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:49.566346    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="config-reloader" containerID="cri-o://4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820" gracePeriod=120
Sep 19 11:36:49.566515 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:49.566378    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="kube-rbac-proxy-metric" containerID="cri-o://b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81" gracePeriod=120
Sep 19 11:36:49.566515 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:49.566340    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="kube-rbac-proxy-web" containerID="cri-o://ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6" gracePeriod=120
Sep 19 11:36:49.566515 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:49.566356    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="prom-label-proxy" containerID="cri-o://00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f" gracePeriod=120
Sep 19 11:36:50.388703 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.388666    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerID="00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f" exitCode=0
Sep 19 11:36:50.388703 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.388698    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerID="c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661" exitCode=0
Sep 19 11:36:50.388703 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.388711    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerID="4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820" exitCode=0
Sep 19 11:36:50.388986 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.388720    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerID="0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a" exitCode=0
Sep 19 11:36:50.388986 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.388740    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb1e3e58-c987-4673-8bb5-bfb72d011d62","Type":"ContainerDied","Data":"00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f"}
Sep 19 11:36:50.388986 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.388819    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb1e3e58-c987-4673-8bb5-bfb72d011d62","Type":"ContainerDied","Data":"c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661"}
Sep 19 11:36:50.388986 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.388861    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb1e3e58-c987-4673-8bb5-bfb72d011d62","Type":"ContainerDied","Data":"4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820"}
Sep 19 11:36:50.388986 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.388896    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb1e3e58-c987-4673-8bb5-bfb72d011d62","Type":"ContainerDied","Data":"0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a"}
Sep 19 11:36:50.909372 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.909135    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:50.956048 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.956013    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdc7k\" (UniqueName: \"kubernetes.io/projected/fb1e3e58-c987-4673-8bb5-bfb72d011d62-kube-api-access-gdc7k\") pod \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") "
Sep 19 11:36:50.956241 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.956057    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-cluster-tls-config\") pod \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") "
Sep 19 11:36:50.956241 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.956090    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb1e3e58-c987-4673-8bb5-bfb72d011d62-tls-assets\") pod \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") "
Sep 19 11:36:50.956241 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.956133    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy\") pod \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") "
Sep 19 11:36:50.956241 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.956174    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb1e3e58-c987-4673-8bb5-bfb72d011d62-alertmanager-main-db\") pod \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") "
Sep 19 11:36:50.956241 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.956232    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb1e3e58-c987-4673-8bb5-bfb72d011d62-config-out\") pod \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") "
Sep 19 11:36:50.956514 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.956257    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-main-tls\") pod \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") "
Sep 19 11:36:50.956514 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.956289    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-web-config\") pod \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") "
Sep 19 11:36:50.956514 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.956321    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1e3e58-c987-4673-8bb5-bfb72d011d62-alertmanager-trusted-ca-bundle\") pod \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") "
Sep 19 11:36:50.956514 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.956350    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy-metric\") pod \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") "
Sep 19 11:36:50.956514 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.956392    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy-web\") pod \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") "
Sep 19 11:36:50.956514 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.956419    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb1e3e58-c987-4673-8bb5-bfb72d011d62-metrics-client-ca\") pod \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") "
Sep 19 11:36:50.956514 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.956455    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-config-volume\") pod \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\" (UID: \"fb1e3e58-c987-4673-8bb5-bfb72d011d62\") "
Sep 19 11:36:50.957016 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.956966    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb1e3e58-c987-4673-8bb5-bfb72d011d62-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "fb1e3e58-c987-4673-8bb5-bfb72d011d62" (UID: "fb1e3e58-c987-4673-8bb5-bfb72d011d62"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:36:50.958387 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.957678    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb1e3e58-c987-4673-8bb5-bfb72d011d62-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "fb1e3e58-c987-4673-8bb5-bfb72d011d62" (UID: "fb1e3e58-c987-4673-8bb5-bfb72d011d62"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:36:50.958387 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.957795    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb1e3e58-c987-4673-8bb5-bfb72d011d62-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "fb1e3e58-c987-4673-8bb5-bfb72d011d62" (UID: "fb1e3e58-c987-4673-8bb5-bfb72d011d62"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:36:50.959138 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.959112    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb1e3e58-c987-4673-8bb5-bfb72d011d62-config-out" (OuterVolumeSpecName: "config-out") pod "fb1e3e58-c987-4673-8bb5-bfb72d011d62" (UID: "fb1e3e58-c987-4673-8bb5-bfb72d011d62"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:36:50.959390 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.959369    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-config-volume" (OuterVolumeSpecName: "config-volume") pod "fb1e3e58-c987-4673-8bb5-bfb72d011d62" (UID: "fb1e3e58-c987-4673-8bb5-bfb72d011d62"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:36:50.960820 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.960793    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "fb1e3e58-c987-4673-8bb5-bfb72d011d62" (UID: "fb1e3e58-c987-4673-8bb5-bfb72d011d62"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:36:50.960917 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.960887    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "fb1e3e58-c987-4673-8bb5-bfb72d011d62" (UID: "fb1e3e58-c987-4673-8bb5-bfb72d011d62"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:36:50.961119 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.961093    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb1e3e58-c987-4673-8bb5-bfb72d011d62-kube-api-access-gdc7k" (OuterVolumeSpecName: "kube-api-access-gdc7k") pod "fb1e3e58-c987-4673-8bb5-bfb72d011d62" (UID: "fb1e3e58-c987-4673-8bb5-bfb72d011d62"). InnerVolumeSpecName "kube-api-access-gdc7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:36:50.961331 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.961284    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "fb1e3e58-c987-4673-8bb5-bfb72d011d62" (UID: "fb1e3e58-c987-4673-8bb5-bfb72d011d62"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:36:50.961947 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.961913    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb1e3e58-c987-4673-8bb5-bfb72d011d62-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "fb1e3e58-c987-4673-8bb5-bfb72d011d62" (UID: "fb1e3e58-c987-4673-8bb5-bfb72d011d62"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:36:50.962861 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.962835    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "fb1e3e58-c987-4673-8bb5-bfb72d011d62" (UID: "fb1e3e58-c987-4673-8bb5-bfb72d011d62"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:36:50.966345 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.966311    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "fb1e3e58-c987-4673-8bb5-bfb72d011d62" (UID: "fb1e3e58-c987-4673-8bb5-bfb72d011d62"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:36:50.973066 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:50.972982    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-web-config" (OuterVolumeSpecName: "web-config") pod "fb1e3e58-c987-4673-8bb5-bfb72d011d62" (UID: "fb1e3e58-c987-4673-8bb5-bfb72d011d62"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:36:51.057257 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.057230    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:51.057257 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.057253    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:51.057412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.057264    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb1e3e58-c987-4673-8bb5-bfb72d011d62-metrics-client-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:51.057412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.057273    2559 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:51.057412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.057282    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdc7k\" (UniqueName: \"kubernetes.io/projected/fb1e3e58-c987-4673-8bb5-bfb72d011d62-kube-api-access-gdc7k\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:51.057412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.057290    2559 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-cluster-tls-config\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:51.057412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.057298    2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb1e3e58-c987-4673-8bb5-bfb72d011d62-tls-assets\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:51.057412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.057306    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:51.057412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.057313    2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb1e3e58-c987-4673-8bb5-bfb72d011d62-alertmanager-main-db\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:51.057412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.057321    2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb1e3e58-c987-4673-8bb5-bfb72d011d62-config-out\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:51.057412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.057330    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-secret-alertmanager-main-tls\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:51.057412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.057337    2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb1e3e58-c987-4673-8bb5-bfb72d011d62-web-config\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:51.057412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.057344    2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1e3e58-c987-4673-8bb5-bfb72d011d62-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:36:51.395660 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.395629    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerID="b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81" exitCode=0
Sep 19 11:36:51.395660 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.395653    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerID="ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6" exitCode=0
Sep 19 11:36:51.395850 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.395682    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb1e3e58-c987-4673-8bb5-bfb72d011d62","Type":"ContainerDied","Data":"b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81"}
Sep 19 11:36:51.395850 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.395701    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.395850 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.395736    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb1e3e58-c987-4673-8bb5-bfb72d011d62","Type":"ContainerDied","Data":"ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6"}
Sep 19 11:36:51.395850 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.395764    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb1e3e58-c987-4673-8bb5-bfb72d011d62","Type":"ContainerDied","Data":"c5baf2b8899bf1be97ed5509bf0509200b08aa338ea85678b15823ef4c8f7561"}
Sep 19 11:36:51.395850 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.395792    2559 scope.go:117] "RemoveContainer" containerID="00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f"
Sep 19 11:36:51.403780 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.403630    2559 scope.go:117] "RemoveContainer" containerID="b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81"
Sep 19 11:36:51.412291 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.412275    2559 scope.go:117] "RemoveContainer" containerID="c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661"
Sep 19 11:36:51.417946 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.417925    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 19 11:36:51.419637 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.419622    2559 scope.go:117] "RemoveContainer" containerID="ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6"
Sep 19 11:36:51.424295 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.424276    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 19 11:36:51.426502 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.426489    2559 scope.go:117] "RemoveContainer" containerID="4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820"
Sep 19 11:36:51.433144 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.433015    2559 scope.go:117] "RemoveContainer" containerID="0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a"
Sep 19 11:36:51.440106 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.440083    2559 scope.go:117] "RemoveContainer" containerID="03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead"
Sep 19 11:36:51.446808 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.446790    2559 scope.go:117] "RemoveContainer" containerID="00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f"
Sep 19 11:36:51.447230 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:36:51.447144    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f\": container with ID starting with 00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f not found: ID does not exist" containerID="00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f"
Sep 19 11:36:51.447339 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.447180    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f"} err="failed to get container status \"00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f\": rpc error: code = NotFound desc = could not find container \"00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f\": container with ID starting with 00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f not found: ID does not exist"
Sep 19 11:36:51.447339 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.447253    2559 scope.go:117] "RemoveContainer" containerID="b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81"
Sep 19 11:36:51.447572 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:36:51.447551    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81\": container with ID starting with b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81 not found: ID does not exist" containerID="b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81"
Sep 19 11:36:51.447719 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.447582    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81"} err="failed to get container status \"b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81\": rpc error: code = NotFound desc = could not find container \"b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81\": container with ID starting with b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81 not found: ID does not exist"
Sep 19 11:36:51.447719 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.447603    2559 scope.go:117] "RemoveContainer" containerID="c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661"
Sep 19 11:36:51.448029 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:36:51.448008    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661\": container with ID starting with c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661 not found: ID does not exist" containerID="c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661"
Sep 19 11:36:51.448109 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.448035    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661"} err="failed to get container status \"c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661\": rpc error: code = NotFound desc = could not find container \"c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661\": container with ID starting with c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661 not found: ID does not exist"
Sep 19 11:36:51.448109 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.448050    2559 scope.go:117] "RemoveContainer" containerID="ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6"
Sep 19 11:36:51.448443 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:36:51.448424    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6\": container with ID starting with ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6 not found: ID does not exist" containerID="ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6"
Sep 19 11:36:51.448522 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.448448    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6"} err="failed to get container status \"ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6\": rpc error: code = NotFound desc = could not find container \"ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6\": container with ID starting with ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6 not found: ID does not exist"
Sep 19 11:36:51.448522 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.448465    2559 scope.go:117] "RemoveContainer" containerID="4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820"
Sep 19 11:36:51.448640 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.448622    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 19 11:36:51.448749 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:36:51.448729    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820\": container with ID starting with 4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820 not found: ID does not exist" containerID="4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820"
Sep 19 11:36:51.448803 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.448756    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820"} err="failed to get container status \"4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820\": rpc error: code = NotFound desc = could not find container \"4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820\": container with ID starting with 4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820 not found: ID does not exist"
Sep 19 11:36:51.448857 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.448802    2559 scope.go:117] "RemoveContainer" containerID="0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a"
Sep 19 11:36:51.449030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449012    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="kube-rbac-proxy-metric"
Sep 19 11:36:51.449030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449030    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="kube-rbac-proxy-metric"
Sep 19 11:36:51.449205 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449046    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="kube-rbac-proxy"
Sep 19 11:36:51.449205 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449054    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="kube-rbac-proxy"
Sep 19 11:36:51.449205 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449072    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfbe13bc-b73b-4b57-bede-50e74ee50d57" containerName="console"
Sep 19 11:36:51.449205 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449079    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfbe13bc-b73b-4b57-bede-50e74ee50d57" containerName="console"
Sep 19 11:36:51.449205 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449089    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="alertmanager"
Sep 19 11:36:51.449205 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449098    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="alertmanager"
Sep 19 11:36:51.449205 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449106    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="config-reloader"
Sep 19 11:36:51.449205 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449114    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="config-reloader"
Sep 19 11:36:51.449205 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449144    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="init-config-reloader"
Sep 19 11:36:51.449205 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449153    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="init-config-reloader"
Sep 19 11:36:51.449205 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449167    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70bed969-3852-4859-9d55-a1546255bf3e" containerName="console"
Sep 19 11:36:51.449205 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449175    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="70bed969-3852-4859-9d55-a1546255bf3e" containerName="console"
Sep 19 11:36:51.449205 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:36:51.449108    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a\": container with ID starting with 0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a not found: ID does not exist" containerID="0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449212    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="prom-label-proxy"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449221    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="prom-label-proxy"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449223    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a"} err="failed to get container status \"0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a\": rpc error: code = NotFound desc = could not find container \"0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a\": container with ID starting with 0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a not found: ID does not exist"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449235    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="kube-rbac-proxy-web"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449238    2559 scope.go:117] "RemoveContainer" containerID="03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449244    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="kube-rbac-proxy-web"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449323    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="prom-label-proxy"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449337    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="kube-rbac-proxy"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449347    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="config-reloader"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449359    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="70bed969-3852-4859-9d55-a1546255bf3e" containerName="console"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449368    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="kube-rbac-proxy-web"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449376    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="kube-rbac-proxy-metric"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449385    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfbe13bc-b73b-4b57-bede-50e74ee50d57" containerName="console"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449392    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" containerName="alertmanager"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:36:51.449494    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead\": container with ID starting with 03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead not found: ID does not exist" containerID="03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449511    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead"} err="failed to get container status \"03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead\": rpc error: code = NotFound desc = could not find container \"03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead\": container with ID starting with 03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead not found: ID does not exist"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449523    2559 scope.go:117] "RemoveContainer" containerID="00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449745    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f"} err="failed to get container status \"00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f\": rpc error: code = NotFound desc = could not find container \"00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f\": container with ID starting with 00187077681e5556fddfcddc4da613a49626286c8c0d2031753d0e959dbf6a3f not found: ID does not exist"
Sep 19 11:36:51.449903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.449771    2559 scope.go:117] "RemoveContainer" containerID="b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81"
Sep 19 11:36:51.450778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.450050    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81"} err="failed to get container status \"b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81\": rpc error: code = NotFound desc = could not find container \"b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81\": container with ID starting with b09f569dc31c8de388f152dad888a8a45effb156400ce41b927b530339838a81 not found: ID does not exist"
Sep 19 11:36:51.450778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.450069    2559 scope.go:117] "RemoveContainer" containerID="c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661"
Sep 19 11:36:51.450778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.450331    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661"} err="failed to get container status \"c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661\": rpc error: code = NotFound desc = could not find container \"c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661\": container with ID starting with c7c5b3731f5d066a328663bddb6cbd2b8871214b5a52e339aea1342134572661 not found: ID does not exist"
Sep 19 11:36:51.450778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.450347    2559 scope.go:117] "RemoveContainer" containerID="ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6"
Sep 19 11:36:51.450778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.450565    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6"} err="failed to get container status \"ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6\": rpc error: code = NotFound desc = could not find container \"ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6\": container with ID starting with ceff45549053fe3d38115842be9f5486b3a30a6b37ef46879b20624d7d28f2d6 not found: ID does not exist"
Sep 19 11:36:51.450778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.450579    2559 scope.go:117] "RemoveContainer" containerID="4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820"
Sep 19 11:36:51.450778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.450767    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820"} err="failed to get container status \"4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820\": rpc error: code = NotFound desc = could not find container \"4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820\": container with ID starting with 4570db0008ec2fa949c10387fd5a06c6d9a9c0e3cb81a8bf7383035561eb8820 not found: ID does not exist"
Sep 19 11:36:51.450778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.450778    2559 scope.go:117] "RemoveContainer" containerID="0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a"
Sep 19 11:36:51.451168 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.450990    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a"} err="failed to get container status \"0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a\": rpc error: code = NotFound desc = could not find container \"0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a\": container with ID starting with 0c8f4237bb294aff425b40bf676c35349f9a08e07d779041b7b5ff7e02cec63a not found: ID does not exist"
Sep 19 11:36:51.451168 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.451005    2559 scope.go:117] "RemoveContainer" containerID="03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead"
Sep 19 11:36:51.451373 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.451294    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead"} err="failed to get container status \"03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead\": rpc error: code = NotFound desc = could not find container \"03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead\": container with ID starting with 03d36c85d1ac11623506a897f36fd7bf6ae7e0b1a1ab133588e925a8369deead not found: ID does not exist"
Sep 19 11:36:51.455017 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.454999    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.457267 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.457246    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 19 11:36:51.457456 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.457439    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 19 11:36:51.457553 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.457477    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 19 11:36:51.457553 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.457480    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 19 11:36:51.457553 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.457500    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 19 11:36:51.457818 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.457801    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kr57b\""
Sep 19 11:36:51.457901 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.457886    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 19 11:36:51.457954 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.457887    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 19 11:36:51.458014 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.458001    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 19 11:36:51.459821 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.459785    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.459903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.459861    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-config-volume\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.459903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.459887    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.460026 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.459918    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-config-out\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.460026 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.459963    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.460026 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.460003    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.460173 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.460027    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.460173 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.460051    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.460173 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.460074    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.460173 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.460110    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.460173 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.460127    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9lnx\" (UniqueName: \"kubernetes.io/projected/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-kube-api-access-x9lnx\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.460455 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.460228    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-web-config\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.460455 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.460251    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.463509 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.463488    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 19 11:36:51.466615 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.466594    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 19 11:36:51.561053 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.561022    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-web-config\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.561053 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.561049    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.561337 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.561069    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.561337 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.561094    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-config-volume\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.561337 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.561114    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.561337 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.561143    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-config-out\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.561337 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.561168    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.561337 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.561224    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.561337 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.561253    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.561337 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.561280    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.561337 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.561298    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.562730 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.561348    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.562730 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.561364    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9lnx\" (UniqueName: \"kubernetes.io/projected/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-kube-api-access-x9lnx\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.562730 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.562261    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.562730 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.562425    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.562730 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.562672    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.564335 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.564290    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.564514 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.564489    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-web-config\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.564514 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.564508    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.564809 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.564785    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.565163 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.565140    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.565266 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.565208    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-config-volume\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.565401 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.565386    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-config-out\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.565433 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.565409    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.568479 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.566184    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.570618 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.570595    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9lnx\" (UniqueName: \"kubernetes.io/projected/6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f-kube-api-access-x9lnx\") pod \"alertmanager-main-0\" (UID: \"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:51.764782 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:51.764751    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 19 11:36:52.042444 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:52.042411    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 19 11:36:52.045892 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:36:52.045861    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e64f5b2_dcd4_4333_ba77_3ddbdd0d5b6f.slice/crio-f5eab8ac192beb33d9d02239d318cf4bec7c6c68644a0c2bdf61f625085dc6ff WatchSource:0}: Error finding container f5eab8ac192beb33d9d02239d318cf4bec7c6c68644a0c2bdf61f625085dc6ff: Status 404 returned error can't find the container with id f5eab8ac192beb33d9d02239d318cf4bec7c6c68644a0c2bdf61f625085dc6ff
Sep 19 11:36:52.401893 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:52.401809    2559 generic.go:356] "Generic (PLEG): container finished" podID="6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f" containerID="eee8725c2bb70f6be23419e96524cc2bf7b02adcd62c135c49bf9aaaff3b43fe" exitCode=0
Sep 19 11:36:52.401893 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:52.401859    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f","Type":"ContainerDied","Data":"eee8725c2bb70f6be23419e96524cc2bf7b02adcd62c135c49bf9aaaff3b43fe"}
Sep 19 11:36:52.402061 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:52.401923    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f","Type":"ContainerStarted","Data":"f5eab8ac192beb33d9d02239d318cf4bec7c6c68644a0c2bdf61f625085dc6ff"}
Sep 19 11:36:52.596125 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:52.596103    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb1e3e58-c987-4673-8bb5-bfb72d011d62" path="/var/lib/kubelet/pods/fb1e3e58-c987-4673-8bb5-bfb72d011d62/volumes"
Sep 19 11:36:53.409207 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.409158    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f","Type":"ContainerStarted","Data":"b5d6bc5b72dc8ace4744bb9e23006d9f04081ef95c26ac9d4bc48089518b6727"}
Sep 19 11:36:53.409572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.409266    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f","Type":"ContainerStarted","Data":"1b45c8c5147bc601c1dfbe93d81bee804b01a61151bafd03aab46a1f7840fbd6"}
Sep 19 11:36:53.409572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.409315    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f","Type":"ContainerStarted","Data":"100847d85f191a39e610d44646c9ba0f40cb101c3244a2592f35960bf4daa817"}
Sep 19 11:36:53.409572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.409350    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f","Type":"ContainerStarted","Data":"dd1b1c05ee4d51622202a3c8fadb2589575837e07988f2ce9a68943e859a64fb"}
Sep 19 11:36:53.409572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.409385    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f","Type":"ContainerStarted","Data":"536e4a1d9ab9e7f2bd908c3a6d7ea19e82a83d7f17001aa66e6db240b7e6b4dd"}
Sep 19 11:36:53.409572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.409426    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f","Type":"ContainerStarted","Data":"cbe2ed0c1558c2fc7700d41ea1ff00b90556e8f99cd3b3af337599e6d19ed067"}
Sep 19 11:36:53.433445 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.433394    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.417309654 podStartE2EDuration="2.433380872s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.071218ms" isStatefulPod=false podCreationTimestamp="2026-09-19 11:36:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:36:53.432308984 +0000 UTC m=+165.354621863" watchObservedRunningTime="2026-09-19 11:36:53.433380872 +0000 UTC m=+165.355693743"
Sep 19 11:36:53.663289 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.663207    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b8cf4dd6d-tvq6s"]
Sep 19 11:36:53.666551 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.666523    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.677779 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.677758    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2kwv\" (UniqueName: \"kubernetes.io/projected/96dccbf0-5feb-4523-b5f7-709f5673c581-kube-api-access-n2kwv\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.677909 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.677794    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-oauth-serving-cert\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.677909 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.677833    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-console-config\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.677909 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.677864    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-trusted-ca-bundle\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.678076 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.677897    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96dccbf0-5feb-4523-b5f7-709f5673c581-console-oauth-config\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.678076 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.677975    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-service-ca\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.678076 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.677984    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b8cf4dd6d-tvq6s"]
Sep 19 11:36:53.678076 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.678004    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96dccbf0-5feb-4523-b5f7-709f5673c581-console-serving-cert\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.778962 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.778923    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2kwv\" (UniqueName: \"kubernetes.io/projected/96dccbf0-5feb-4523-b5f7-709f5673c581-kube-api-access-n2kwv\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.779148 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.778976    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-oauth-serving-cert\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.779148 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.779012    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-console-config\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.779148 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.779040    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-trusted-ca-bundle\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.779148 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.779069    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96dccbf0-5feb-4523-b5f7-709f5673c581-console-oauth-config\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.779148 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.779116    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-service-ca\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.779148 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.779141    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96dccbf0-5feb-4523-b5f7-709f5673c581-console-serving-cert\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.780546 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.780507    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-console-config\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.780750 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.780731    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-service-ca\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.780843 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.780827    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-oauth-serving-cert\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.780967 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.780944    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-trusted-ca-bundle\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.782755 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.782731    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96dccbf0-5feb-4523-b5f7-709f5673c581-console-oauth-config\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.782853 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.782740    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96dccbf0-5feb-4523-b5f7-709f5673c581-console-serving-cert\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.788886 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.788864    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2kwv\" (UniqueName: \"kubernetes.io/projected/96dccbf0-5feb-4523-b5f7-709f5673c581-kube-api-access-n2kwv\") pod \"console-6b8cf4dd6d-tvq6s\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") " pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:53.977586 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:53.977556    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:36:54.256859 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:54.256784    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b8cf4dd6d-tvq6s"]
Sep 19 11:36:54.260025 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:36:54.259986    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96dccbf0_5feb_4523_b5f7_709f5673c581.slice/crio-63bf5dcc3783bf3b1e1abdc4bee4bcf2c4bac6b8e85b09c988617a2d9fb20827 WatchSource:0}: Error finding container 63bf5dcc3783bf3b1e1abdc4bee4bcf2c4bac6b8e85b09c988617a2d9fb20827: Status 404 returned error can't find the container with id 63bf5dcc3783bf3b1e1abdc4bee4bcf2c4bac6b8e85b09c988617a2d9fb20827
Sep 19 11:36:54.419079 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:54.419041    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8cf4dd6d-tvq6s" event={"ID":"96dccbf0-5feb-4523-b5f7-709f5673c581","Type":"ContainerStarted","Data":"d11ae55a0b90e132767cd9cb14c82953a59d3b0a6400325a003367e4071926e8"}
Sep 19 11:36:54.419537 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:54.419110    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8cf4dd6d-tvq6s" event={"ID":"96dccbf0-5feb-4523-b5f7-709f5673c581","Type":"ContainerStarted","Data":"63bf5dcc3783bf3b1e1abdc4bee4bcf2c4bac6b8e85b09c988617a2d9fb20827"}
Sep 19 11:36:54.436182 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:36:54.436136    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b8cf4dd6d-tvq6s" podStartSLOduration=1.436122345 podStartE2EDuration="1.436122345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:36:54.434584953 +0000 UTC m=+166.356897862" watchObservedRunningTime="2026-09-19 11:36:54.436122345 +0000 UTC m=+166.358435226"
Sep 19 11:37:03.977933 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:03.977899    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:37:03.978370 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:03.977961    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:37:03.982518 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:03.982498    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:37:04.461508 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:04.461471    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:37:04.506589 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:04.506554    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dfc6fbdf4-ffs6p"]
Sep 19 11:37:29.530526 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.530465    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6dfc6fbdf4-ffs6p" podUID="c0f82a6a-fd78-495f-be13-5626f133e611" containerName="console" containerID="cri-o://f4b1aa62965734c610da617a98862e68ec1d7c5e75b8ff1aef65e04f50eafd4e" gracePeriod=15
Sep 19 11:37:29.769613 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.769587    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:37:29.877447 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.877375    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0f82a6a-fd78-495f-be13-5626f133e611-console-oauth-config\") pod \"c0f82a6a-fd78-495f-be13-5626f133e611\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") "
Sep 19 11:37:29.877447 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.877409    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-oauth-serving-cert\") pod \"c0f82a6a-fd78-495f-be13-5626f133e611\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") "
Sep 19 11:37:29.877447 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.877427    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0f82a6a-fd78-495f-be13-5626f133e611-console-serving-cert\") pod \"c0f82a6a-fd78-495f-be13-5626f133e611\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") "
Sep 19 11:37:29.877687 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.877464    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-service-ca\") pod \"c0f82a6a-fd78-495f-be13-5626f133e611\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") "
Sep 19 11:37:29.877687 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.877531    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-trusted-ca-bundle\") pod \"c0f82a6a-fd78-495f-be13-5626f133e611\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") "
Sep 19 11:37:29.877687 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.877591    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-console-config\") pod \"c0f82a6a-fd78-495f-be13-5626f133e611\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") "
Sep 19 11:37:29.877687 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.877623    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxgzb\" (UniqueName: \"kubernetes.io/projected/c0f82a6a-fd78-495f-be13-5626f133e611-kube-api-access-jxgzb\") pod \"c0f82a6a-fd78-495f-be13-5626f133e611\" (UID: \"c0f82a6a-fd78-495f-be13-5626f133e611\") "
Sep 19 11:37:29.877887 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.877827    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c0f82a6a-fd78-495f-be13-5626f133e611" (UID: "c0f82a6a-fd78-495f-be13-5626f133e611"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:37:29.878009 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.877974    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-console-config" (OuterVolumeSpecName: "console-config") pod "c0f82a6a-fd78-495f-be13-5626f133e611" (UID: "c0f82a6a-fd78-495f-be13-5626f133e611"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:37:29.878071 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.878002    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c0f82a6a-fd78-495f-be13-5626f133e611" (UID: "c0f82a6a-fd78-495f-be13-5626f133e611"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:37:29.878149 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.878133    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-service-ca" (OuterVolumeSpecName: "service-ca") pod "c0f82a6a-fd78-495f-be13-5626f133e611" (UID: "c0f82a6a-fd78-495f-be13-5626f133e611"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:37:29.879880 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.879860    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0f82a6a-fd78-495f-be13-5626f133e611-kube-api-access-jxgzb" (OuterVolumeSpecName: "kube-api-access-jxgzb") pod "c0f82a6a-fd78-495f-be13-5626f133e611" (UID: "c0f82a6a-fd78-495f-be13-5626f133e611"). InnerVolumeSpecName "kube-api-access-jxgzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:37:29.880290 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.880270    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0f82a6a-fd78-495f-be13-5626f133e611-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c0f82a6a-fd78-495f-be13-5626f133e611" (UID: "c0f82a6a-fd78-495f-be13-5626f133e611"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:37:29.880349 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.880295    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0f82a6a-fd78-495f-be13-5626f133e611-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c0f82a6a-fd78-495f-be13-5626f133e611" (UID: "c0f82a6a-fd78-495f-be13-5626f133e611"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:37:29.978581 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.978551    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0f82a6a-fd78-495f-be13-5626f133e611-console-oauth-config\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:37:29.978581 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.978577    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-oauth-serving-cert\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:37:29.978581 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.978586    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0f82a6a-fd78-495f-be13-5626f133e611-console-serving-cert\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:37:29.978798 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.978595    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:37:29.978798 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.978603    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-trusted-ca-bundle\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:37:29.978798 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.978611    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0f82a6a-fd78-495f-be13-5626f133e611-console-config\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:37:29.978798 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:29.978619    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxgzb\" (UniqueName: \"kubernetes.io/projected/c0f82a6a-fd78-495f-be13-5626f133e611-kube-api-access-jxgzb\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:37:30.556809 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:30.556773    2559 generic.go:356] "Generic (PLEG): container finished" podID="c0f82a6a-fd78-495f-be13-5626f133e611" containerID="f4b1aa62965734c610da617a98862e68ec1d7c5e75b8ff1aef65e04f50eafd4e" exitCode=0
Sep 19 11:37:30.557247 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:30.556825    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfc6fbdf4-ffs6p"
Sep 19 11:37:30.557247 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:30.556847    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfc6fbdf4-ffs6p" event={"ID":"c0f82a6a-fd78-495f-be13-5626f133e611","Type":"ContainerDied","Data":"f4b1aa62965734c610da617a98862e68ec1d7c5e75b8ff1aef65e04f50eafd4e"}
Sep 19 11:37:30.557247 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:30.556912    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfc6fbdf4-ffs6p" event={"ID":"c0f82a6a-fd78-495f-be13-5626f133e611","Type":"ContainerDied","Data":"3e3c8bbb0bf729b04767cced37e054366ad20d1ffc01976dcb4f48098e5be74e"}
Sep 19 11:37:30.557247 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:30.556932    2559 scope.go:117] "RemoveContainer" containerID="f4b1aa62965734c610da617a98862e68ec1d7c5e75b8ff1aef65e04f50eafd4e"
Sep 19 11:37:30.568506 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:30.568490    2559 scope.go:117] "RemoveContainer" containerID="f4b1aa62965734c610da617a98862e68ec1d7c5e75b8ff1aef65e04f50eafd4e"
Sep 19 11:37:30.568752 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:37:30.568730    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4b1aa62965734c610da617a98862e68ec1d7c5e75b8ff1aef65e04f50eafd4e\": container with ID starting with f4b1aa62965734c610da617a98862e68ec1d7c5e75b8ff1aef65e04f50eafd4e not found: ID does not exist" containerID="f4b1aa62965734c610da617a98862e68ec1d7c5e75b8ff1aef65e04f50eafd4e"
Sep 19 11:37:30.568807 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:30.568760    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b1aa62965734c610da617a98862e68ec1d7c5e75b8ff1aef65e04f50eafd4e"} err="failed to get container status \"f4b1aa62965734c610da617a98862e68ec1d7c5e75b8ff1aef65e04f50eafd4e\": rpc error: code = NotFound desc = could not find container \"f4b1aa62965734c610da617a98862e68ec1d7c5e75b8ff1aef65e04f50eafd4e\": container with ID starting with f4b1aa62965734c610da617a98862e68ec1d7c5e75b8ff1aef65e04f50eafd4e not found: ID does not exist"
Sep 19 11:37:30.576112 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:30.576091    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dfc6fbdf4-ffs6p"]
Sep 19 11:37:30.579942 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:30.579918    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6dfc6fbdf4-ffs6p"]
Sep 19 11:37:30.593429 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:30.593405    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0f82a6a-fd78-495f-be13-5626f133e611" path="/var/lib/kubelet/pods/c0f82a6a-fd78-495f-be13-5626f133e611/volumes"
Sep 19 11:37:48.307098 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.307067    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"]
Sep 19 11:37:48.307573 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.307417    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f82a6a-fd78-495f-be13-5626f133e611" containerName="console"
Sep 19 11:37:48.307573 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.307428    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f82a6a-fd78-495f-be13-5626f133e611" containerName="console"
Sep 19 11:37:48.307573 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.307474    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0f82a6a-fd78-495f-be13-5626f133e611" containerName="console"
Sep 19 11:37:48.311836 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.311818    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"
Sep 19 11:37:48.314006 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.313983    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 11:37:48.314837 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.314816    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-f9x6f\""
Sep 19 11:37:48.314961 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.314821    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 11:37:48.319701 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.319677    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"]
Sep 19 11:37:48.320765 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.320744    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86c3ff1a-91cd-4295-802c-a81dfa609466-util\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr\" (UID: \"86c3ff1a-91cd-4295-802c-a81dfa609466\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"
Sep 19 11:37:48.320842 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.320782    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86c3ff1a-91cd-4295-802c-a81dfa609466-bundle\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr\" (UID: \"86c3ff1a-91cd-4295-802c-a81dfa609466\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"
Sep 19 11:37:48.320842 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.320810    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfph4\" (UniqueName: \"kubernetes.io/projected/86c3ff1a-91cd-4295-802c-a81dfa609466-kube-api-access-nfph4\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr\" (UID: \"86c3ff1a-91cd-4295-802c-a81dfa609466\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"
Sep 19 11:37:48.422115 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.422083    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86c3ff1a-91cd-4295-802c-a81dfa609466-util\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr\" (UID: \"86c3ff1a-91cd-4295-802c-a81dfa609466\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"
Sep 19 11:37:48.422115 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.422119    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86c3ff1a-91cd-4295-802c-a81dfa609466-bundle\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr\" (UID: \"86c3ff1a-91cd-4295-802c-a81dfa609466\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"
Sep 19 11:37:48.422382 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.422142    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfph4\" (UniqueName: \"kubernetes.io/projected/86c3ff1a-91cd-4295-802c-a81dfa609466-kube-api-access-nfph4\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr\" (UID: \"86c3ff1a-91cd-4295-802c-a81dfa609466\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"
Sep 19 11:37:48.422542 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.422520    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86c3ff1a-91cd-4295-802c-a81dfa609466-util\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr\" (UID: \"86c3ff1a-91cd-4295-802c-a81dfa609466\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"
Sep 19 11:37:48.422610 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.422543    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86c3ff1a-91cd-4295-802c-a81dfa609466-bundle\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr\" (UID: \"86c3ff1a-91cd-4295-802c-a81dfa609466\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"
Sep 19 11:37:48.432781 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.432755    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfph4\" (UniqueName: \"kubernetes.io/projected/86c3ff1a-91cd-4295-802c-a81dfa609466-kube-api-access-nfph4\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr\" (UID: \"86c3ff1a-91cd-4295-802c-a81dfa609466\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"
Sep 19 11:37:48.543427 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.543397    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"]
Sep 19 11:37:48.546863 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.546848    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"
Sep 19 11:37:48.558000 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.557936    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"]
Sep 19 11:37:48.622166 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.622138    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"
Sep 19 11:37:48.623632 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.623608    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32dee056-13b9-4574-a5c1-b5fc51afdd12-util\") pod \"978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck\" (UID: \"32dee056-13b9-4574-a5c1-b5fc51afdd12\") " pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"
Sep 19 11:37:48.623742 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.623659    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpdfq\" (UniqueName: \"kubernetes.io/projected/32dee056-13b9-4574-a5c1-b5fc51afdd12-kube-api-access-gpdfq\") pod \"978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck\" (UID: \"32dee056-13b9-4574-a5c1-b5fc51afdd12\") " pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"
Sep 19 11:37:48.623742 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.623684    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32dee056-13b9-4574-a5c1-b5fc51afdd12-bundle\") pod \"978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck\" (UID: \"32dee056-13b9-4574-a5c1-b5fc51afdd12\") " pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"
Sep 19 11:37:48.724507 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.724478    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32dee056-13b9-4574-a5c1-b5fc51afdd12-util\") pod \"978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck\" (UID: \"32dee056-13b9-4574-a5c1-b5fc51afdd12\") " pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"
Sep 19 11:37:48.724666 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.724537    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpdfq\" (UniqueName: \"kubernetes.io/projected/32dee056-13b9-4574-a5c1-b5fc51afdd12-kube-api-access-gpdfq\") pod \"978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck\" (UID: \"32dee056-13b9-4574-a5c1-b5fc51afdd12\") " pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"
Sep 19 11:37:48.724666 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.724566    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32dee056-13b9-4574-a5c1-b5fc51afdd12-bundle\") pod \"978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck\" (UID: \"32dee056-13b9-4574-a5c1-b5fc51afdd12\") " pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"
Sep 19 11:37:48.724916 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.724885    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32dee056-13b9-4574-a5c1-b5fc51afdd12-util\") pod \"978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck\" (UID: \"32dee056-13b9-4574-a5c1-b5fc51afdd12\") " pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"
Sep 19 11:37:48.725035 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.724899    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32dee056-13b9-4574-a5c1-b5fc51afdd12-bundle\") pod \"978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck\" (UID: \"32dee056-13b9-4574-a5c1-b5fc51afdd12\") " pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"
Sep 19 11:37:48.733073 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.733050    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpdfq\" (UniqueName: \"kubernetes.io/projected/32dee056-13b9-4574-a5c1-b5fc51afdd12-kube-api-access-gpdfq\") pod \"978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck\" (UID: \"32dee056-13b9-4574-a5c1-b5fc51afdd12\") " pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"
Sep 19 11:37:48.817133 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.817054    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"]
Sep 19 11:37:48.821576 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.821555    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:48.823942 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.823919    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 19 11:37:48.824079 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.823919    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 19 11:37:48.824079 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.823993    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 19 11:37:48.824079 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.823919    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 19 11:37:48.824354 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.824338    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 19 11:37:48.824412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.824391    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-5bt8j\""
Sep 19 11:37:48.824459 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.824338    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 11:37:48.834136 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.834117    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"]
Sep 19 11:37:48.855374 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.855354    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"
Sep 19 11:37:48.892865 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.892730    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"]
Sep 19 11:37:48.895834 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:37:48.895803    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86c3ff1a_91cd_4295_802c_a81dfa609466.slice/crio-d0b7fba93d10595e07d08f097e7ec5098a42dec466d93e75168c7c9dce8ae945 WatchSource:0}: Error finding container d0b7fba93d10595e07d08f097e7ec5098a42dec466d93e75168c7c9dce8ae945: Status 404 returned error can't find the container with id d0b7fba93d10595e07d08f097e7ec5098a42dec466d93e75168c7c9dce8ae945
Sep 19 11:37:48.925978 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.925947    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmgh9\" (UniqueName: \"kubernetes.io/projected/f04747b6-1efd-4d06-8068-63f196883961-kube-api-access-wmgh9\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:48.926112 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.926010    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:48.926112 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.926043    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:48.926112 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.926068    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f04747b6-1efd-4d06-8068-63f196883961-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:48.926269 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.926135    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f04747b6-1efd-4d06-8068-63f196883961-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:48.926269 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.926224    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f04747b6-1efd-4d06-8068-63f196883961-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:48.926643 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:48.926270    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.026644 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.026609    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgh9\" (UniqueName: \"kubernetes.io/projected/f04747b6-1efd-4d06-8068-63f196883961-kube-api-access-wmgh9\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.026644 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.026641    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.026867 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.026661    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.026867 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.026681    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f04747b6-1efd-4d06-8068-63f196883961-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.026867 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.026716    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f04747b6-1efd-4d06-8068-63f196883961-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.026867 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.026765    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f04747b6-1efd-4d06-8068-63f196883961-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.026867 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.026801    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.027409 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.027386    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f04747b6-1efd-4d06-8068-63f196883961-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.029227 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.029184    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.029689 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.029663    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f04747b6-1efd-4d06-8068-63f196883961-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.029804 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.029787    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.029870 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.029841    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.034713 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.034694    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmgh9\" (UniqueName: \"kubernetes.io/projected/f04747b6-1efd-4d06-8068-63f196883961-kube-api-access-wmgh9\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.034952 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.034933    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f04747b6-1efd-4d06-8068-63f196883961-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-mmlpb\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.126090 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.126069    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"]
Sep 19 11:37:49.127852 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:37:49.127829    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32dee056_13b9_4574_a5c1_b5fc51afdd12.slice/crio-ea8cb5bdb64c624283a68d8b7d7da7cc924de5c79109d17bc26f807666026f33 WatchSource:0}: Error finding container ea8cb5bdb64c624283a68d8b7d7da7cc924de5c79109d17bc26f807666026f33: Status 404 returned error can't find the container with id ea8cb5bdb64c624283a68d8b7d7da7cc924de5c79109d17bc26f807666026f33
Sep 19 11:37:49.131153 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.131120    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:49.411276 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.411245    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"]
Sep 19 11:37:49.413672 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:37:49.413639    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf04747b6_1efd_4d06_8068_63f196883961.slice/crio-d48125742ee66102a9c47f2e2b0328762a46148813997f2bdae31f16fee0ae10 WatchSource:0}: Error finding container d48125742ee66102a9c47f2e2b0328762a46148813997f2bdae31f16fee0ae10: Status 404 returned error can't find the container with id d48125742ee66102a9c47f2e2b0328762a46148813997f2bdae31f16fee0ae10
Sep 19 11:37:49.639548 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.639038    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr" event={"ID":"86c3ff1a-91cd-4295-802c-a81dfa609466","Type":"ContainerStarted","Data":"d0b7fba93d10595e07d08f097e7ec5098a42dec466d93e75168c7c9dce8ae945"}
Sep 19 11:37:49.640880 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.640854    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb" event={"ID":"f04747b6-1efd-4d06-8068-63f196883961","Type":"ContainerStarted","Data":"d48125742ee66102a9c47f2e2b0328762a46148813997f2bdae31f16fee0ae10"}
Sep 19 11:37:49.642519 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:49.642494    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck" event={"ID":"32dee056-13b9-4574-a5c1-b5fc51afdd12","Type":"ContainerStarted","Data":"ea8cb5bdb64c624283a68d8b7d7da7cc924de5c79109d17bc26f807666026f33"}
Sep 19 11:37:52.510266 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:52.510010    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:37:52.510266 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:52.510090    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:37:52.661236 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:52.661165    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb" event={"ID":"f04747b6-1efd-4d06-8068-63f196883961","Type":"ContainerStarted","Data":"6a5b2c370fb9ff622e3fe3277df22bd676c9eeeb71d397e0551445c21d34f6c5"}
Sep 19 11:37:52.681379 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:52.681303    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb" podStartSLOduration=1.587658824 podStartE2EDuration="4.681284302s" totalImagesPullingTime="3.093625478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:37:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:37:52.679179026 +0000 UTC m=+224.601491943" watchObservedRunningTime="2026-09-19 11:37:52.681284302 +0000 UTC m=+224.603597185"
Sep 19 11:37:53.671379 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:53.671315    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:53.673810 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:53.673572    2559 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-mmlpb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 11:37:53.673810 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:53.673793    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb" podUID="f04747b6-1efd-4d06-8068-63f196883961" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 11:37:54.678414 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:54.678381    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:37:55.683003 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:55.682974    2559 generic.go:356] "Generic (PLEG): container finished" podID="32dee056-13b9-4574-a5c1-b5fc51afdd12" containerID="c5d99e2b16972fec33437f861434b6098fcb1c0b4df232a48475521c9be5e16d" exitCode=0
Sep 19 11:37:55.683472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:55.683068    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck" event={"ID":"32dee056-13b9-4574-a5c1-b5fc51afdd12","Type":"ContainerDied","Data":"c5d99e2b16972fec33437f861434b6098fcb1c0b4df232a48475521c9be5e16d"}
Sep 19 11:37:55.684673 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:55.684649    2559 generic.go:356] "Generic (PLEG): container finished" podID="86c3ff1a-91cd-4295-802c-a81dfa609466" containerID="3db24073fb8be6b3cd3e685b09e9be5e33aefa2f635d0ff2ed273736b4f8fc22" exitCode=0
Sep 19 11:37:55.684815 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:55.684710    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr" event={"ID":"86c3ff1a-91cd-4295-802c-a81dfa609466","Type":"ContainerDied","Data":"3db24073fb8be6b3cd3e685b09e9be5e33aefa2f635d0ff2ed273736b4f8fc22"}
Sep 19 11:37:56.690849 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:56.690821    2559 generic.go:356] "Generic (PLEG): container finished" podID="32dee056-13b9-4574-a5c1-b5fc51afdd12" containerID="e21dbdeed1e7043f20adf04391e5cebfb1b2e1dbfb1f5e9414a9053cb0c7eb70" exitCode=0
Sep 19 11:37:56.691215 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:56.690909    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck" event={"ID":"32dee056-13b9-4574-a5c1-b5fc51afdd12","Type":"ContainerDied","Data":"e21dbdeed1e7043f20adf04391e5cebfb1b2e1dbfb1f5e9414a9053cb0c7eb70"}
Sep 19 11:37:58.703351 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:58.703313    2559 generic.go:356] "Generic (PLEG): container finished" podID="86c3ff1a-91cd-4295-802c-a81dfa609466" containerID="dbf8ea8b43f0ac32fd4afcc4743e5e9e06a3773214c97d6cd99850dd7bc4cbcb" exitCode=0
Sep 19 11:37:58.703786 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:37:58.703365    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr" event={"ID":"86c3ff1a-91cd-4295-802c-a81dfa609466","Type":"ContainerDied","Data":"dbf8ea8b43f0ac32fd4afcc4743e5e9e06a3773214c97d6cd99850dd7bc4cbcb"}
Sep 19 11:38:04.736104 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:04.736072    2559 generic.go:356] "Generic (PLEG): container finished" podID="32dee056-13b9-4574-a5c1-b5fc51afdd12" containerID="bd4e6b081eca9d2100cc13eb963cad2629440977aa4c66131dfb4d8689562212" exitCode=0
Sep 19 11:38:04.736627 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:04.736159    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck" event={"ID":"32dee056-13b9-4574-a5c1-b5fc51afdd12","Type":"ContainerDied","Data":"bd4e6b081eca9d2100cc13eb963cad2629440977aa4c66131dfb4d8689562212"}
Sep 19 11:38:04.738202 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:04.738165    2559 generic.go:356] "Generic (PLEG): container finished" podID="86c3ff1a-91cd-4295-802c-a81dfa609466" containerID="36272c29bff778ee35a54b55804973503e1d2b3cd11731794448a4fc4bfc12b1" exitCode=0
Sep 19 11:38:04.738308 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:04.738220    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr" event={"ID":"86c3ff1a-91cd-4295-802c-a81dfa609466","Type":"ContainerDied","Data":"36272c29bff778ee35a54b55804973503e1d2b3cd11731794448a4fc4bfc12b1"}
Sep 19 11:38:05.890416 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:05.890393    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"
Sep 19 11:38:05.893127 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:05.893108    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"
Sep 19 11:38:05.982629 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:05.982595    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86c3ff1a-91cd-4295-802c-a81dfa609466-util\") pod \"86c3ff1a-91cd-4295-802c-a81dfa609466\" (UID: \"86c3ff1a-91cd-4295-802c-a81dfa609466\") "
Sep 19 11:38:05.982817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:05.982661    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpdfq\" (UniqueName: \"kubernetes.io/projected/32dee056-13b9-4574-a5c1-b5fc51afdd12-kube-api-access-gpdfq\") pod \"32dee056-13b9-4574-a5c1-b5fc51afdd12\" (UID: \"32dee056-13b9-4574-a5c1-b5fc51afdd12\") "
Sep 19 11:38:05.982817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:05.982712    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86c3ff1a-91cd-4295-802c-a81dfa609466-bundle\") pod \"86c3ff1a-91cd-4295-802c-a81dfa609466\" (UID: \"86c3ff1a-91cd-4295-802c-a81dfa609466\") "
Sep 19 11:38:05.982817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:05.982733    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32dee056-13b9-4574-a5c1-b5fc51afdd12-bundle\") pod \"32dee056-13b9-4574-a5c1-b5fc51afdd12\" (UID: \"32dee056-13b9-4574-a5c1-b5fc51afdd12\") "
Sep 19 11:38:05.982817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:05.982768    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfph4\" (UniqueName: \"kubernetes.io/projected/86c3ff1a-91cd-4295-802c-a81dfa609466-kube-api-access-nfph4\") pod \"86c3ff1a-91cd-4295-802c-a81dfa609466\" (UID: \"86c3ff1a-91cd-4295-802c-a81dfa609466\") "
Sep 19 11:38:05.982817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:05.982791    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32dee056-13b9-4574-a5c1-b5fc51afdd12-util\") pod \"32dee056-13b9-4574-a5c1-b5fc51afdd12\" (UID: \"32dee056-13b9-4574-a5c1-b5fc51afdd12\") "
Sep 19 11:38:05.983209 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:05.983164    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c3ff1a-91cd-4295-802c-a81dfa609466-bundle" (OuterVolumeSpecName: "bundle") pod "86c3ff1a-91cd-4295-802c-a81dfa609466" (UID: "86c3ff1a-91cd-4295-802c-a81dfa609466"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:38:05.983628 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:05.983480    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32dee056-13b9-4574-a5c1-b5fc51afdd12-bundle" (OuterVolumeSpecName: "bundle") pod "32dee056-13b9-4574-a5c1-b5fc51afdd12" (UID: "32dee056-13b9-4574-a5c1-b5fc51afdd12"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:38:05.985109 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:05.985086    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32dee056-13b9-4574-a5c1-b5fc51afdd12-kube-api-access-gpdfq" (OuterVolumeSpecName: "kube-api-access-gpdfq") pod "32dee056-13b9-4574-a5c1-b5fc51afdd12" (UID: "32dee056-13b9-4574-a5c1-b5fc51afdd12"). InnerVolumeSpecName "kube-api-access-gpdfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:38:05.985221 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:05.985119    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86c3ff1a-91cd-4295-802c-a81dfa609466-kube-api-access-nfph4" (OuterVolumeSpecName: "kube-api-access-nfph4") pod "86c3ff1a-91cd-4295-802c-a81dfa609466" (UID: "86c3ff1a-91cd-4295-802c-a81dfa609466"). InnerVolumeSpecName "kube-api-access-nfph4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:38:05.987358 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:05.987338    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32dee056-13b9-4574-a5c1-b5fc51afdd12-util" (OuterVolumeSpecName: "util") pod "32dee056-13b9-4574-a5c1-b5fc51afdd12" (UID: "32dee056-13b9-4574-a5c1-b5fc51afdd12"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:38:05.988489 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:05.988466    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c3ff1a-91cd-4295-802c-a81dfa609466-util" (OuterVolumeSpecName: "util") pod "86c3ff1a-91cd-4295-802c-a81dfa609466" (UID: "86c3ff1a-91cd-4295-802c-a81dfa609466"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:38:06.083722 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:06.083650    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32dee056-13b9-4574-a5c1-b5fc51afdd12-util\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:38:06.083722 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:06.083680    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86c3ff1a-91cd-4295-802c-a81dfa609466-util\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:38:06.083722 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:06.083689    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpdfq\" (UniqueName: \"kubernetes.io/projected/32dee056-13b9-4574-a5c1-b5fc51afdd12-kube-api-access-gpdfq\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:38:06.083722 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:06.083699    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86c3ff1a-91cd-4295-802c-a81dfa609466-bundle\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:38:06.083722 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:06.083709    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32dee056-13b9-4574-a5c1-b5fc51afdd12-bundle\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:38:06.083722 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:06.083716    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfph4\" (UniqueName: \"kubernetes.io/projected/86c3ff1a-91cd-4295-802c-a81dfa609466-kube-api-access-nfph4\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:38:06.748170 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:06.748139    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck"
Sep 19 11:38:06.748370 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:06.748146    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/978bd81eed4c4dc7896cbc7c21ae91125ef0d2dcd9abf7d17f5d8e954ecwqck" event={"ID":"32dee056-13b9-4574-a5c1-b5fc51afdd12","Type":"ContainerDied","Data":"ea8cb5bdb64c624283a68d8b7d7da7cc924de5c79109d17bc26f807666026f33"}
Sep 19 11:38:06.748370 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:06.748245    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea8cb5bdb64c624283a68d8b7d7da7cc924de5c79109d17bc26f807666026f33"
Sep 19 11:38:06.750543 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:06.750516    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr" event={"ID":"86c3ff1a-91cd-4295-802c-a81dfa609466","Type":"ContainerDied","Data":"d0b7fba93d10595e07d08f097e7ec5098a42dec466d93e75168c7c9dce8ae945"}
Sep 19 11:38:06.750666 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:06.750553    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0b7fba93d10595e07d08f097e7ec5098a42dec466d93e75168c7c9dce8ae945"
Sep 19 11:38:06.750666 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:06.750571    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78evjbwr"
Sep 19 11:38:11.871552 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.871519    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"]
Sep 19 11:38:11.871934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.871812    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86c3ff1a-91cd-4295-802c-a81dfa609466" containerName="util"
Sep 19 11:38:11.871934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.871823    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="86c3ff1a-91cd-4295-802c-a81dfa609466" containerName="util"
Sep 19 11:38:11.871934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.871833    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32dee056-13b9-4574-a5c1-b5fc51afdd12" containerName="extract"
Sep 19 11:38:11.871934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.871838    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="32dee056-13b9-4574-a5c1-b5fc51afdd12" containerName="extract"
Sep 19 11:38:11.871934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.871853    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32dee056-13b9-4574-a5c1-b5fc51afdd12" containerName="pull"
Sep 19 11:38:11.871934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.871858    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="32dee056-13b9-4574-a5c1-b5fc51afdd12" containerName="pull"
Sep 19 11:38:11.871934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.871865    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86c3ff1a-91cd-4295-802c-a81dfa609466" containerName="pull"
Sep 19 11:38:11.871934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.871871    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="86c3ff1a-91cd-4295-802c-a81dfa609466" containerName="pull"
Sep 19 11:38:11.871934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.871879    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32dee056-13b9-4574-a5c1-b5fc51afdd12" containerName="util"
Sep 19 11:38:11.871934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.871883    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="32dee056-13b9-4574-a5c1-b5fc51afdd12" containerName="util"
Sep 19 11:38:11.871934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.871893    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86c3ff1a-91cd-4295-802c-a81dfa609466" containerName="extract"
Sep 19 11:38:11.871934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.871897    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="86c3ff1a-91cd-4295-802c-a81dfa609466" containerName="extract"
Sep 19 11:38:11.872305 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.871943    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="86c3ff1a-91cd-4295-802c-a81dfa609466" containerName="extract"
Sep 19 11:38:11.872305 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.871951    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="32dee056-13b9-4574-a5c1-b5fc51afdd12" containerName="extract"
Sep 19 11:38:11.876141 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.876126    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:11.881022 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.880991    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 19 11:38:11.881184 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.881056    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 19 11:38:11.881357 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.881259    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Sep 19 11:38:11.881357 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.881333    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Sep 19 11:38:11.882248 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.882210    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Sep 19 11:38:11.882248 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.882232    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-cjw94\""
Sep 19 11:38:11.890819 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.890791    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"]
Sep 19 11:38:11.934293 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.934263    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f75bdb2c-1911-4576-89d3-aec9a72b5d17-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:11.934293 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.934298    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f75bdb2c-1911-4576-89d3-aec9a72b5d17-metrics-cert\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:11.934463 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.934365    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbmx8\" (UniqueName: \"kubernetes.io/projected/f75bdb2c-1911-4576-89d3-aec9a72b5d17-kube-api-access-tbmx8\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:11.934463 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.934390    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f75bdb2c-1911-4576-89d3-aec9a72b5d17-webhook-cert\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:11.934463 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.934448    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f75bdb2c-1911-4576-89d3-aec9a72b5d17-manifests\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:11.934557 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:11.934498    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f75bdb2c-1911-4576-89d3-aec9a72b5d17-tmp\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:12.035303 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:12.035274    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f75bdb2c-1911-4576-89d3-aec9a72b5d17-tmp\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:12.035303 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:12.035306    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f75bdb2c-1911-4576-89d3-aec9a72b5d17-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:12.035541 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:12.035325    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f75bdb2c-1911-4576-89d3-aec9a72b5d17-metrics-cert\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:12.035541 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:12.035379    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbmx8\" (UniqueName: \"kubernetes.io/projected/f75bdb2c-1911-4576-89d3-aec9a72b5d17-kube-api-access-tbmx8\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:12.035541 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:12.035410    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f75bdb2c-1911-4576-89d3-aec9a72b5d17-webhook-cert\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:12.035541 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:12.035439    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f75bdb2c-1911-4576-89d3-aec9a72b5d17-manifests\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:12.035837 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:12.035806    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f75bdb2c-1911-4576-89d3-aec9a72b5d17-manifests\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:12.037784 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:12.037759    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f75bdb2c-1911-4576-89d3-aec9a72b5d17-tmp\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:12.038034 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:12.038013    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f75bdb2c-1911-4576-89d3-aec9a72b5d17-metrics-cert\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:12.038214 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:12.038170    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f75bdb2c-1911-4576-89d3-aec9a72b5d17-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:12.038261 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:12.038177    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f75bdb2c-1911-4576-89d3-aec9a72b5d17-webhook-cert\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:12.054176 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:12.054157    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbmx8\" (UniqueName: \"kubernetes.io/projected/f75bdb2c-1911-4576-89d3-aec9a72b5d17-kube-api-access-tbmx8\") pod \"opendatahub-operator-controller-manager-9f59f68b4-q2742\" (UID: \"f75bdb2c-1911-4576-89d3-aec9a72b5d17\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:12.186390 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:12.186359    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:12.462603 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:12.462580    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"]
Sep 19 11:38:12.465065 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:38:12.465040    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf75bdb2c_1911_4576_89d3_aec9a72b5d17.slice/crio-f5535508f84fc538992507911c8fdb2a1415dc14d8eaa23ecbc253c910e5f7c6 WatchSource:0}: Error finding container f5535508f84fc538992507911c8fdb2a1415dc14d8eaa23ecbc253c910e5f7c6: Status 404 returned error can't find the container with id f5535508f84fc538992507911c8fdb2a1415dc14d8eaa23ecbc253c910e5f7c6
Sep 19 11:38:12.779408 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:12.779331    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742" event={"ID":"f75bdb2c-1911-4576-89d3-aec9a72b5d17","Type":"ContainerStarted","Data":"f5535508f84fc538992507911c8fdb2a1415dc14d8eaa23ecbc253c910e5f7c6"}
Sep 19 11:38:15.796847 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:15.796820    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742" event={"ID":"f75bdb2c-1911-4576-89d3-aec9a72b5d17","Type":"ContainerStarted","Data":"753784ae7998aae47fae1303294392e312aa72f7802a4996e46bba48e7d0f4b3"}
Sep 19 11:38:16.804493 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:16.804452    2559 generic.go:356] "Generic (PLEG): container finished" podID="f75bdb2c-1911-4576-89d3-aec9a72b5d17" containerID="753784ae7998aae47fae1303294392e312aa72f7802a4996e46bba48e7d0f4b3" exitCode=0
Sep 19 11:38:16.804493 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:16.804532    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742" event={"ID":"f75bdb2c-1911-4576-89d3-aec9a72b5d17","Type":"ContainerDied","Data":"753784ae7998aae47fae1303294392e312aa72f7802a4996e46bba48e7d0f4b3"}
Sep 19 11:38:16.805101 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:16.804588    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742" event={"ID":"f75bdb2c-1911-4576-89d3-aec9a72b5d17","Type":"ContainerStarted","Data":"70bf68e87b51b5140c02a5d03ffafdc0a0439dcbaf168c0ea7a87c2303a001ee"}
Sep 19 11:38:16.805101 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:16.804618    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:16.827948 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:16.827898    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742" podStartSLOduration=2.197849125 podStartE2EDuration="5.827880087s" totalImagesPullingTime="3.159135994s" totalInitContainerRuntime="470.894968ms" isStatefulPod=false podCreationTimestamp="2026-09-19 11:38:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:38:16.827036519 +0000 UTC m=+248.749349400" watchObservedRunningTime="2026-09-19 11:38:16.827880087 +0000 UTC m=+248.750192968"
Sep 19 11:38:27.811062 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:27.811036    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9f59f68b4-q2742"
Sep 19 11:38:39.406890 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.406799    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"]
Sep 19 11:38:39.411571 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.411548    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:38:39.414223 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.414203    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 19 11:38:39.414467 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.414233    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 19 11:38:39.414467 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.414247    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Sep 19 11:38:39.415114 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.415096    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-l9tzn\""
Sep 19 11:38:39.422098 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.422076    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"]
Sep 19 11:38:39.567495 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.567466    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwvst\" (UniqueName: \"kubernetes.io/projected/c8344758-3571-45a9-bf9b-1c191dd357f5-kube-api-access-hwvst\") pod \"kserve-module-controller-manager-54765d59fd-45g7g\" (UID: \"c8344758-3571-45a9-bf9b-1c191dd357f5\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:38:39.567663 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.567507    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c8344758-3571-45a9-bf9b-1c191dd357f5-trusted-ca\") pod \"kserve-module-controller-manager-54765d59fd-45g7g\" (UID: \"c8344758-3571-45a9-bf9b-1c191dd357f5\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:38:39.567663 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.567584    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8344758-3571-45a9-bf9b-1c191dd357f5-tmp\") pod \"kserve-module-controller-manager-54765d59fd-45g7g\" (UID: \"c8344758-3571-45a9-bf9b-1c191dd357f5\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:38:39.567663 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.567624    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c8344758-3571-45a9-bf9b-1c191dd357f5-manifests\") pod \"kserve-module-controller-manager-54765d59fd-45g7g\" (UID: \"c8344758-3571-45a9-bf9b-1c191dd357f5\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:38:39.668942 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.668864    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwvst\" (UniqueName: \"kubernetes.io/projected/c8344758-3571-45a9-bf9b-1c191dd357f5-kube-api-access-hwvst\") pod \"kserve-module-controller-manager-54765d59fd-45g7g\" (UID: \"c8344758-3571-45a9-bf9b-1c191dd357f5\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:38:39.668942 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.668904    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c8344758-3571-45a9-bf9b-1c191dd357f5-trusted-ca\") pod \"kserve-module-controller-manager-54765d59fd-45g7g\" (UID: \"c8344758-3571-45a9-bf9b-1c191dd357f5\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:38:39.669172 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.669148    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8344758-3571-45a9-bf9b-1c191dd357f5-tmp\") pod \"kserve-module-controller-manager-54765d59fd-45g7g\" (UID: \"c8344758-3571-45a9-bf9b-1c191dd357f5\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:38:39.669324 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.669223    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c8344758-3571-45a9-bf9b-1c191dd357f5-manifests\") pod \"kserve-module-controller-manager-54765d59fd-45g7g\" (UID: \"c8344758-3571-45a9-bf9b-1c191dd357f5\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:38:39.669522 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.669502    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8344758-3571-45a9-bf9b-1c191dd357f5-tmp\") pod \"kserve-module-controller-manager-54765d59fd-45g7g\" (UID: \"c8344758-3571-45a9-bf9b-1c191dd357f5\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:38:39.669582 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.669564    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c8344758-3571-45a9-bf9b-1c191dd357f5-manifests\") pod \"kserve-module-controller-manager-54765d59fd-45g7g\" (UID: \"c8344758-3571-45a9-bf9b-1c191dd357f5\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:38:39.669782 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.669759    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c8344758-3571-45a9-bf9b-1c191dd357f5-trusted-ca\") pod \"kserve-module-controller-manager-54765d59fd-45g7g\" (UID: \"c8344758-3571-45a9-bf9b-1c191dd357f5\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:38:39.677949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.677931    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwvst\" (UniqueName: \"kubernetes.io/projected/c8344758-3571-45a9-bf9b-1c191dd357f5-kube-api-access-hwvst\") pod \"kserve-module-controller-manager-54765d59fd-45g7g\" (UID: \"c8344758-3571-45a9-bf9b-1c191dd357f5\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:38:39.723420 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:39.723397    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:38:40.000354 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:40.000326    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"]
Sep 19 11:38:40.002740 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:38:40.002706    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8344758_3571_45a9_bf9b_1c191dd357f5.slice/crio-ddfbafda8eb42b578fc07df589e0571b099139ed2c9c5dbfbf761073c766bc29 WatchSource:0}: Error finding container ddfbafda8eb42b578fc07df589e0571b099139ed2c9c5dbfbf761073c766bc29: Status 404 returned error can't find the container with id ddfbafda8eb42b578fc07df589e0571b099139ed2c9c5dbfbf761073c766bc29
Sep 19 11:38:40.928984 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:40.928903    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g" event={"ID":"c8344758-3571-45a9-bf9b-1c191dd357f5","Type":"ContainerStarted","Data":"ddfbafda8eb42b578fc07df589e0571b099139ed2c9c5dbfbf761073c766bc29"}
Sep 19 11:38:41.544735 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:41.544697    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sttnj"]
Sep 19 11:38:41.553726 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:41.553687    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sttnj"
Sep 19 11:38:41.556698 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:41.556500    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-wcc4q\""
Sep 19 11:38:41.556698 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:41.556523    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 19 11:38:41.556698 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:41.556500    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 19 11:38:41.557477 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:41.557429    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sttnj"]
Sep 19 11:38:41.588873 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:41.588848    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdwrz\" (UniqueName: \"kubernetes.io/projected/c3aadd1e-6acc-4b36-81f8-4f38a0aedb05-kube-api-access-qdwrz\") pod \"cert-manager-55b4d954d5-sttnj\" (UID: \"c3aadd1e-6acc-4b36-81f8-4f38a0aedb05\") " pod="cert-manager/cert-manager-55b4d954d5-sttnj"
Sep 19 11:38:41.589080 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:41.589049    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3aadd1e-6acc-4b36-81f8-4f38a0aedb05-bound-sa-token\") pod \"cert-manager-55b4d954d5-sttnj\" (UID: \"c3aadd1e-6acc-4b36-81f8-4f38a0aedb05\") " pod="cert-manager/cert-manager-55b4d954d5-sttnj"
Sep 19 11:38:41.690224 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:41.690140    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdwrz\" (UniqueName: \"kubernetes.io/projected/c3aadd1e-6acc-4b36-81f8-4f38a0aedb05-kube-api-access-qdwrz\") pod \"cert-manager-55b4d954d5-sttnj\" (UID: \"c3aadd1e-6acc-4b36-81f8-4f38a0aedb05\") " pod="cert-manager/cert-manager-55b4d954d5-sttnj"
Sep 19 11:38:41.690493 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:41.690420    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3aadd1e-6acc-4b36-81f8-4f38a0aedb05-bound-sa-token\") pod \"cert-manager-55b4d954d5-sttnj\" (UID: \"c3aadd1e-6acc-4b36-81f8-4f38a0aedb05\") " pod="cert-manager/cert-manager-55b4d954d5-sttnj"
Sep 19 11:38:41.704113 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:41.704076    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdwrz\" (UniqueName: \"kubernetes.io/projected/c3aadd1e-6acc-4b36-81f8-4f38a0aedb05-kube-api-access-qdwrz\") pod \"cert-manager-55b4d954d5-sttnj\" (UID: \"c3aadd1e-6acc-4b36-81f8-4f38a0aedb05\") " pod="cert-manager/cert-manager-55b4d954d5-sttnj"
Sep 19 11:38:41.704463 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:41.704444    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3aadd1e-6acc-4b36-81f8-4f38a0aedb05-bound-sa-token\") pod \"cert-manager-55b4d954d5-sttnj\" (UID: \"c3aadd1e-6acc-4b36-81f8-4f38a0aedb05\") " pod="cert-manager/cert-manager-55b4d954d5-sttnj"
Sep 19 11:38:41.872479 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:41.872401    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sttnj"
Sep 19 11:38:42.595800 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:42.595772    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sttnj"]
Sep 19 11:38:42.597007 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:38:42.596983    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3aadd1e_6acc_4b36_81f8_4f38a0aedb05.slice/crio-820ce556a448fd3d83e9c8a6e4ae6ec8d27aea7747d8c4c2cfd109bcd8797138 WatchSource:0}: Error finding container 820ce556a448fd3d83e9c8a6e4ae6ec8d27aea7747d8c4c2cfd109bcd8797138: Status 404 returned error can't find the container with id 820ce556a448fd3d83e9c8a6e4ae6ec8d27aea7747d8c4c2cfd109bcd8797138
Sep 19 11:38:42.939283 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:42.939250    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g" event={"ID":"c8344758-3571-45a9-bf9b-1c191dd357f5","Type":"ContainerStarted","Data":"51a9d252b6d63092f27600a8f2e4863f515c89d6bb77337ef0a8b73bd0eaf310"}
Sep 19 11:38:42.939455 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:42.939378    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:38:42.940533 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:42.940509    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sttnj" event={"ID":"c3aadd1e-6acc-4b36-81f8-4f38a0aedb05","Type":"ContainerStarted","Data":"820ce556a448fd3d83e9c8a6e4ae6ec8d27aea7747d8c4c2cfd109bcd8797138"}
Sep 19 11:38:42.957250 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:42.957214    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g" podStartSLOduration=1.5692293990000001 podStartE2EDuration="3.957184771s" totalImagesPullingTime="2.387955372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:38:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:38:42.956110896 +0000 UTC m=+274.878423790" watchObservedRunningTime="2026-09-19 11:38:42.957184771 +0000 UTC m=+274.879497651"
Sep 19 11:38:43.201956 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:43.201882    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lndfg"]
Sep 19 11:38:43.205612 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:43.205590    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lndfg"
Sep 19 11:38:43.207895 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:43.207877    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-5kzh9\""
Sep 19 11:38:43.214729 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:43.214698    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lndfg"]
Sep 19 11:38:43.305573 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:43.305540    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eaba5c28-e442-4280-bb64-41712426fce8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lndfg\" (UID: \"eaba5c28-e442-4280-bb64-41712426fce8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lndfg"
Sep 19 11:38:43.305758 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:43.305603    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7gdj\" (UniqueName: \"kubernetes.io/projected/eaba5c28-e442-4280-bb64-41712426fce8-kube-api-access-t7gdj\") pod \"cert-manager-cainjector-559cdd8c87-lndfg\" (UID: \"eaba5c28-e442-4280-bb64-41712426fce8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lndfg"
Sep 19 11:38:43.406743 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:43.406707    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eaba5c28-e442-4280-bb64-41712426fce8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lndfg\" (UID: \"eaba5c28-e442-4280-bb64-41712426fce8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lndfg"
Sep 19 11:38:43.406916 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:43.406782    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7gdj\" (UniqueName: \"kubernetes.io/projected/eaba5c28-e442-4280-bb64-41712426fce8-kube-api-access-t7gdj\") pod \"cert-manager-cainjector-559cdd8c87-lndfg\" (UID: \"eaba5c28-e442-4280-bb64-41712426fce8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lndfg"
Sep 19 11:38:43.415180 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:43.415130    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eaba5c28-e442-4280-bb64-41712426fce8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lndfg\" (UID: \"eaba5c28-e442-4280-bb64-41712426fce8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lndfg"
Sep 19 11:38:43.415582 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:43.415549    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7gdj\" (UniqueName: \"kubernetes.io/projected/eaba5c28-e442-4280-bb64-41712426fce8-kube-api-access-t7gdj\") pod \"cert-manager-cainjector-559cdd8c87-lndfg\" (UID: \"eaba5c28-e442-4280-bb64-41712426fce8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lndfg"
Sep 19 11:38:43.518446 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:43.518324    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lndfg"
Sep 19 11:38:43.805404 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:43.805296    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lndfg"]
Sep 19 11:38:43.948615 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:43.948048    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lndfg" event={"ID":"eaba5c28-e442-4280-bb64-41712426fce8","Type":"ContainerStarted","Data":"69f5d7dd28e01caf49ce38b683bcb543f10ca731738c65ce81f43302396b3290"}
Sep 19 11:38:45.959777 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:45.959678    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sttnj" event={"ID":"c3aadd1e-6acc-4b36-81f8-4f38a0aedb05","Type":"ContainerStarted","Data":"5d021c46b9dac5d238601651779313d86ee517b0263ec68911b41b0e325cdb38"}
Sep 19 11:38:45.961580 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:45.961552    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lndfg" event={"ID":"eaba5c28-e442-4280-bb64-41712426fce8","Type":"ContainerStarted","Data":"9afb23c1e42579f40d5e244e694f6b7e32f682bf787ad11bf97b84f8263479a9"}
Sep 19 11:38:45.975694 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:45.975652    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-sttnj" podStartSLOduration=1.8655044200000002 podStartE2EDuration="4.975636967s" totalImagesPullingTime="3.110132547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:38:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:38:45.97496036 +0000 UTC m=+277.897273243" watchObservedRunningTime="2026-09-19 11:38:45.975636967 +0000 UTC m=+277.897949847"
Sep 19 11:38:45.996640 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:38:45.996478    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lndfg" podStartSLOduration=1.10063929 podStartE2EDuration="2.996458081s" totalImagesPullingTime="1.895818791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:38:45.996072433 +0000 UTC m=+277.918385314" watchObservedRunningTime="2026-09-19 11:38:45.996458081 +0000 UTC m=+277.918770962"
Sep 19 11:39:03.949811 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:03.949778    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-54765d59fd-45g7g"
Sep 19 11:39:08.491305 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:08.491267    2559 kubelet.go:1648] "Image garbage collection succeeded"
Sep 19 11:39:30.195258 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.195223    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tpt4s"]
Sep 19 11:39:30.199801 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.199779    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tpt4s"
Sep 19 11:39:30.204150 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.204103    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 19 11:39:30.204376 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.204359    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 19 11:39:30.205082 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.205036    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 19 11:39:30.205300 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.205279    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qq8p4\""
Sep 19 11:39:30.225334 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.225308    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tpt4s"]
Sep 19 11:39:30.302578 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.302543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w62kv\" (UniqueName: \"kubernetes.io/projected/ea50d270-9b5e-4bbc-ab20-d22db3ba9d78-kube-api-access-w62kv\") pod \"limitador-limitador-8588749977-tpt4s\" (UID: \"ea50d270-9b5e-4bbc-ab20-d22db3ba9d78\") " pod="kuadrant-system/limitador-limitador-8588749977-tpt4s"
Sep 19 11:39:30.302761 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.302689    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ea50d270-9b5e-4bbc-ab20-d22db3ba9d78-config-file\") pod \"limitador-limitador-8588749977-tpt4s\" (UID: \"ea50d270-9b5e-4bbc-ab20-d22db3ba9d78\") " pod="kuadrant-system/limitador-limitador-8588749977-tpt4s"
Sep 19 11:39:30.303478 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.303448    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tpt4s"]
Sep 19 11:39:30.366316 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.366289    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv"]
Sep 19 11:39:30.369623 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.369601    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv"
Sep 19 11:39:30.374202 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.374166    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 19 11:39:30.375514 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.375495    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 19 11:39:30.390455 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.390434    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv"]
Sep 19 11:39:30.403736 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.403710    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w62kv\" (UniqueName: \"kubernetes.io/projected/ea50d270-9b5e-4bbc-ab20-d22db3ba9d78-kube-api-access-w62kv\") pod \"limitador-limitador-8588749977-tpt4s\" (UID: \"ea50d270-9b5e-4bbc-ab20-d22db3ba9d78\") " pod="kuadrant-system/limitador-limitador-8588749977-tpt4s"
Sep 19 11:39:30.403849 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.403820    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ea50d270-9b5e-4bbc-ab20-d22db3ba9d78-config-file\") pod \"limitador-limitador-8588749977-tpt4s\" (UID: \"ea50d270-9b5e-4bbc-ab20-d22db3ba9d78\") " pod="kuadrant-system/limitador-limitador-8588749977-tpt4s"
Sep 19 11:39:30.404453 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.404434    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ea50d270-9b5e-4bbc-ab20-d22db3ba9d78-config-file\") pod \"limitador-limitador-8588749977-tpt4s\" (UID: \"ea50d270-9b5e-4bbc-ab20-d22db3ba9d78\") " pod="kuadrant-system/limitador-limitador-8588749977-tpt4s"
Sep 19 11:39:30.418690 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.418669    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w62kv\" (UniqueName: \"kubernetes.io/projected/ea50d270-9b5e-4bbc-ab20-d22db3ba9d78-kube-api-access-w62kv\") pod \"limitador-limitador-8588749977-tpt4s\" (UID: \"ea50d270-9b5e-4bbc-ab20-d22db3ba9d78\") " pod="kuadrant-system/limitador-limitador-8588749977-tpt4s"
Sep 19 11:39:30.505295 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.505235    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d1f84f0-c532-4981-aa15-dbed60d05e18-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-thdmv\" (UID: \"3d1f84f0-c532-4981-aa15-dbed60d05e18\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv"
Sep 19 11:39:30.505404 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.505318    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3d1f84f0-c532-4981-aa15-dbed60d05e18-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-thdmv\" (UID: \"3d1f84f0-c532-4981-aa15-dbed60d05e18\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv"
Sep 19 11:39:30.505404 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.505338    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mpm9\" (UniqueName: \"kubernetes.io/projected/3d1f84f0-c532-4981-aa15-dbed60d05e18-kube-api-access-6mpm9\") pod \"kuadrant-console-plugin-fbdd4f445-thdmv\" (UID: \"3d1f84f0-c532-4981-aa15-dbed60d05e18\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv"
Sep 19 11:39:30.512286 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.512269    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tpt4s"
Sep 19 11:39:30.606099 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.606069    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d1f84f0-c532-4981-aa15-dbed60d05e18-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-thdmv\" (UID: \"3d1f84f0-c532-4981-aa15-dbed60d05e18\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv"
Sep 19 11:39:30.606268 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.606135    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3d1f84f0-c532-4981-aa15-dbed60d05e18-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-thdmv\" (UID: \"3d1f84f0-c532-4981-aa15-dbed60d05e18\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv"
Sep 19 11:39:30.606268 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.606158    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mpm9\" (UniqueName: \"kubernetes.io/projected/3d1f84f0-c532-4981-aa15-dbed60d05e18-kube-api-access-6mpm9\") pod \"kuadrant-console-plugin-fbdd4f445-thdmv\" (UID: \"3d1f84f0-c532-4981-aa15-dbed60d05e18\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv"
Sep 19 11:39:30.606268 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:39:30.606221    2559 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 19 11:39:30.606384 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:39:30.606283    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d1f84f0-c532-4981-aa15-dbed60d05e18-plugin-serving-cert podName:3d1f84f0-c532-4981-aa15-dbed60d05e18 nodeName:}" failed. No retries permitted until 2026-09-19 11:39:31.106266917 +0000 UTC m=+323.028579788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/3d1f84f0-c532-4981-aa15-dbed60d05e18-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-thdmv" (UID: "3d1f84f0-c532-4981-aa15-dbed60d05e18") : secret "plugin-serving-cert" not found
Sep 19 11:39:30.606828 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.606810    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3d1f84f0-c532-4981-aa15-dbed60d05e18-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-thdmv\" (UID: \"3d1f84f0-c532-4981-aa15-dbed60d05e18\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv"
Sep 19 11:39:30.615568 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.615544    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mpm9\" (UniqueName: \"kubernetes.io/projected/3d1f84f0-c532-4981-aa15-dbed60d05e18-kube-api-access-6mpm9\") pod \"kuadrant-console-plugin-fbdd4f445-thdmv\" (UID: \"3d1f84f0-c532-4981-aa15-dbed60d05e18\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv"
Sep 19 11:39:30.786337 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.786316    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tpt4s"]
Sep 19 11:39:30.788595 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:39:30.788562    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea50d270_9b5e_4bbc_ab20_d22db3ba9d78.slice/crio-62d9b4b82502cbaced1e8abfad456315c3afb72bcdc4ac4f8d314627ea61e23d WatchSource:0}: Error finding container 62d9b4b82502cbaced1e8abfad456315c3afb72bcdc4ac4f8d314627ea61e23d: Status 404 returned error can't find the container with id 62d9b4b82502cbaced1e8abfad456315c3afb72bcdc4ac4f8d314627ea61e23d
Sep 19 11:39:30.790684 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:30.790668    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 11:39:31.112009 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.111933    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d1f84f0-c532-4981-aa15-dbed60d05e18-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-thdmv\" (UID: \"3d1f84f0-c532-4981-aa15-dbed60d05e18\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv"
Sep 19 11:39:31.114520 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.114497    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d1f84f0-c532-4981-aa15-dbed60d05e18-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-thdmv\" (UID: \"3d1f84f0-c532-4981-aa15-dbed60d05e18\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv"
Sep 19 11:39:31.166558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.166520    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tpt4s" event={"ID":"ea50d270-9b5e-4bbc-ab20-d22db3ba9d78","Type":"ContainerStarted","Data":"62d9b4b82502cbaced1e8abfad456315c3afb72bcdc4ac4f8d314627ea61e23d"}
Sep 19 11:39:31.223521 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.223492    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-tzkmq"]
Sep 19 11:39:31.232882 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.232837    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-tzkmq"
Sep 19 11:39:31.235560 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.235539    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d599g\""
Sep 19 11:39:31.240602 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.240581    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-tzkmq"]
Sep 19 11:39:31.279089 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.279065    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv"
Sep 19 11:39:31.314102 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.314069    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27p67\" (UniqueName: \"kubernetes.io/projected/de84840e-a833-4203-bdef-766036d68ba6-kube-api-access-27p67\") pod \"authorino-7bcfd8f9cd-tzkmq\" (UID: \"de84840e-a833-4203-bdef-766036d68ba6\") " pod="kuadrant-system/authorino-7bcfd8f9cd-tzkmq"
Sep 19 11:39:31.415020 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.414932    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27p67\" (UniqueName: \"kubernetes.io/projected/de84840e-a833-4203-bdef-766036d68ba6-kube-api-access-27p67\") pod \"authorino-7bcfd8f9cd-tzkmq\" (UID: \"de84840e-a833-4203-bdef-766036d68ba6\") " pod="kuadrant-system/authorino-7bcfd8f9cd-tzkmq"
Sep 19 11:39:31.422538 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.422516    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27p67\" (UniqueName: \"kubernetes.io/projected/de84840e-a833-4203-bdef-766036d68ba6-kube-api-access-27p67\") pod \"authorino-7bcfd8f9cd-tzkmq\" (UID: \"de84840e-a833-4203-bdef-766036d68ba6\") " pod="kuadrant-system/authorino-7bcfd8f9cd-tzkmq"
Sep 19 11:39:31.544262 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.544222    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-tzkmq"
Sep 19 11:39:31.561051 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.560999    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv"]
Sep 19 11:39:31.564512 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:39:31.564488    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d1f84f0_c532_4981_aa15_dbed60d05e18.slice/crio-0dfe465d44a32b47582d41c71d8fd34c86a46506daf1c1216e226451689469d3 WatchSource:0}: Error finding container 0dfe465d44a32b47582d41c71d8fd34c86a46506daf1c1216e226451689469d3: Status 404 returned error can't find the container with id 0dfe465d44a32b47582d41c71d8fd34c86a46506daf1c1216e226451689469d3
Sep 19 11:39:31.646004 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.645970    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr"]
Sep 19 11:39:31.650289 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.650265    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr"
Sep 19 11:39:31.652805 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.652786    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 19 11:39:31.653077 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.652777    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-hnnqq\""
Sep 19 11:39:31.668068 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.667970    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr"]
Sep 19 11:39:31.718662 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.718630    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32f38755-5a54-4a59-a523-e8958e46b40f-cert\") pod \"llmisvc-controller-manager-78f7d99c47-22zfr\" (UID: \"32f38755-5a54-4a59-a523-e8958e46b40f\") " pod="opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr"
Sep 19 11:39:31.718833 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.718674    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vmmr\" (UniqueName: \"kubernetes.io/projected/32f38755-5a54-4a59-a523-e8958e46b40f-kube-api-access-8vmmr\") pod \"llmisvc-controller-manager-78f7d99c47-22zfr\" (UID: \"32f38755-5a54-4a59-a523-e8958e46b40f\") " pod="opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr"
Sep 19 11:39:31.819784 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.819316    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32f38755-5a54-4a59-a523-e8958e46b40f-cert\") pod \"llmisvc-controller-manager-78f7d99c47-22zfr\" (UID: \"32f38755-5a54-4a59-a523-e8958e46b40f\") " pod="opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr"
Sep 19 11:39:31.819784 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.819382    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vmmr\" (UniqueName: \"kubernetes.io/projected/32f38755-5a54-4a59-a523-e8958e46b40f-kube-api-access-8vmmr\") pod \"llmisvc-controller-manager-78f7d99c47-22zfr\" (UID: \"32f38755-5a54-4a59-a523-e8958e46b40f\") " pod="opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr"
Sep 19 11:39:31.823370 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.823342    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32f38755-5a54-4a59-a523-e8958e46b40f-cert\") pod \"llmisvc-controller-manager-78f7d99c47-22zfr\" (UID: \"32f38755-5a54-4a59-a523-e8958e46b40f\") " pod="opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr"
Sep 19 11:39:31.833894 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.833846    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vmmr\" (UniqueName: \"kubernetes.io/projected/32f38755-5a54-4a59-a523-e8958e46b40f-kube-api-access-8vmmr\") pod \"llmisvc-controller-manager-78f7d99c47-22zfr\" (UID: \"32f38755-5a54-4a59-a523-e8958e46b40f\") " pod="opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr"
Sep 19 11:39:31.840607 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.840585    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-tzkmq"]
Sep 19 11:39:31.872848 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.872819    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"]
Sep 19 11:39:31.878077 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.878057    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:31.886815 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.886794    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"]
Sep 19 11:39:31.892497 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:39:31.892476    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde84840e_a833_4203_bdef_766036d68ba6.slice/crio-692619d8d15fdc121dda038b4a01c68371847f9ba528c5776356bfe1a7515a58 WatchSource:0}: Error finding container 692619d8d15fdc121dda038b4a01c68371847f9ba528c5776356bfe1a7515a58: Status 404 returned error can't find the container with id 692619d8d15fdc121dda038b4a01c68371847f9ba528c5776356bfe1a7515a58
Sep 19 11:39:31.920137 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.920058    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/890b8972-ec70-408c-8846-fc41234aa2b4-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:31.920137 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.920102    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/890b8972-ec70-408c-8846-fc41234aa2b4-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:31.920384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.920231    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/890b8972-ec70-408c-8846-fc41234aa2b4-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:31.920384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.920273    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/890b8972-ec70-408c-8846-fc41234aa2b4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:31.920384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.920303    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68n9s\" (UniqueName: \"kubernetes.io/projected/890b8972-ec70-408c-8846-fc41234aa2b4-kube-api-access-68n9s\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:31.920384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.920353    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/890b8972-ec70-408c-8846-fc41234aa2b4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:31.920573 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.920450    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/890b8972-ec70-408c-8846-fc41234aa2b4-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:31.968332 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:31.968299    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr"
Sep 19 11:39:32.021787 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.021754    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/890b8972-ec70-408c-8846-fc41234aa2b4-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:32.021787 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.021818    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/890b8972-ec70-408c-8846-fc41234aa2b4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:32.022027 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.021869    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68n9s\" (UniqueName: \"kubernetes.io/projected/890b8972-ec70-408c-8846-fc41234aa2b4-kube-api-access-68n9s\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:32.022027 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.021932    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/890b8972-ec70-408c-8846-fc41234aa2b4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:32.022027 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.021961    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/890b8972-ec70-408c-8846-fc41234aa2b4-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:32.022027 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.021998    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/890b8972-ec70-408c-8846-fc41234aa2b4-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:32.022296 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.022028    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/890b8972-ec70-408c-8846-fc41234aa2b4-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:32.024707 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.024682    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/890b8972-ec70-408c-8846-fc41234aa2b4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:32.026389 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.026305    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/890b8972-ec70-408c-8846-fc41234aa2b4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:32.027135 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.027113    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/890b8972-ec70-408c-8846-fc41234aa2b4-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:32.029423 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.029399    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/890b8972-ec70-408c-8846-fc41234aa2b4-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:32.030337 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.030281    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/890b8972-ec70-408c-8846-fc41234aa2b4-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:32.030871 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.030850    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68n9s\" (UniqueName: \"kubernetes.io/projected/890b8972-ec70-408c-8846-fc41234aa2b4-kube-api-access-68n9s\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:32.031633 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.031589    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/890b8972-ec70-408c-8846-fc41234aa2b4-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-wkt7t\" (UID: \"890b8972-ec70-408c-8846-fc41234aa2b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:32.173151 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.173065    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tpt4s" event={"ID":"ea50d270-9b5e-4bbc-ab20-d22db3ba9d78","Type":"ContainerStarted","Data":"7c79448153cc16f4fce41b7ff719445a28e95c9f5f92163c1f0cda4c7080977b"}
Sep 19 11:39:32.173339 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.173164    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-tpt4s"
Sep 19 11:39:32.174411 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.174390    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv" event={"ID":"3d1f84f0-c532-4981-aa15-dbed60d05e18","Type":"ContainerStarted","Data":"0dfe465d44a32b47582d41c71d8fd34c86a46506daf1c1216e226451689469d3"}
Sep 19 11:39:32.175506 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.175487    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-tzkmq" event={"ID":"de84840e-a833-4203-bdef-766036d68ba6","Type":"ContainerStarted","Data":"692619d8d15fdc121dda038b4a01c68371847f9ba528c5776356bfe1a7515a58"}
Sep 19 11:39:32.194879 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.194858    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:32.210683 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.210635    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-tpt4s" podStartSLOduration=1.043176182 podStartE2EDuration="2.210617959s" totalImagesPullingTime="1.167441777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:39:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:39:32.208753547 +0000 UTC m=+324.131066426" watchObservedRunningTime="2026-09-19 11:39:32.210617959 +0000 UTC m=+324.132930841"
Sep 19 11:39:32.255361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.254578    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr"]
Sep 19 11:39:32.258115 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:39:32.258091    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod32f38755_5a54_4a59_a523_e8958e46b40f.slice/crio-d756c1a5c5e43c5143d650cc5e4f7a36c9a89735e100dcf15b41bbe301d80be7 WatchSource:0}: Error finding container d756c1a5c5e43c5143d650cc5e4f7a36c9a89735e100dcf15b41bbe301d80be7: Status 404 returned error can't find the container with id d756c1a5c5e43c5143d650cc5e4f7a36c9a89735e100dcf15b41bbe301d80be7
Sep 19 11:39:32.477732 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.477650    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"]
Sep 19 11:39:32.482778 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:39:32.482746    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod890b8972_ec70_408c_8846_fc41234aa2b4.slice/crio-7ed25fa622db8e6b7342c3a0e168f2cd064fa48498be9642d1b55e2633da94bf WatchSource:0}: Error finding container 7ed25fa622db8e6b7342c3a0e168f2cd064fa48498be9642d1b55e2633da94bf: Status 404 returned error can't find the container with id 7ed25fa622db8e6b7342c3a0e168f2cd064fa48498be9642d1b55e2633da94bf
Sep 19 11:39:32.487420 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.487376    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:39:32.487562 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:32.487461    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:39:33.187735 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:33.187568    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t" event={"ID":"890b8972-ec70-408c-8846-fc41234aa2b4","Type":"ContainerStarted","Data":"47d8f1f8af69a853d7b69dffea45d58afed9bdfdc7f4f4034a56bef26ee4988c"}
Sep 19 11:39:33.187735 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:33.187658    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:33.187735 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:33.187682    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t" event={"ID":"890b8972-ec70-408c-8846-fc41234aa2b4","Type":"ContainerStarted","Data":"7ed25fa622db8e6b7342c3a0e168f2cd064fa48498be9642d1b55e2633da94bf"}
Sep 19 11:39:33.192663 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:33.192591    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr" event={"ID":"32f38755-5a54-4a59-a523-e8958e46b40f","Type":"ContainerStarted","Data":"d756c1a5c5e43c5143d650cc5e4f7a36c9a89735e100dcf15b41bbe301d80be7"}
Sep 19 11:39:33.204713 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:33.204449    2559 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-wkt7t container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 11:39:33.204713 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:33.204497    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t" podUID="890b8972-ec70-408c-8846-fc41234aa2b4" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 11:39:33.209226 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:33.208304    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t" podStartSLOduration=2.20828553 podStartE2EDuration="2.20828553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:39:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:39:33.205595429 +0000 UTC m=+325.127908311" watchObservedRunningTime="2026-09-19 11:39:33.20828553 +0000 UTC m=+325.130598412"
Sep 19 11:39:34.199635 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:34.199605    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wkt7t"
Sep 19 11:39:34.253452 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:34.253357    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"]
Sep 19 11:39:34.253636 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:34.253571    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb" podUID="f04747b6-1efd-4d06-8068-63f196883961" containerName="discovery" containerID="cri-o://6a5b2c370fb9ff622e3fe3277df22bd676c9eeeb71d397e0551445c21d34f6c5" gracePeriod=30
Sep 19 11:39:35.207885 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:35.207850    2559 generic.go:356] "Generic (PLEG): container finished" podID="f04747b6-1efd-4d06-8068-63f196883961" containerID="6a5b2c370fb9ff622e3fe3277df22bd676c9eeeb71d397e0551445c21d34f6c5" exitCode=0
Sep 19 11:39:35.208399 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:35.207928    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb" event={"ID":"f04747b6-1efd-4d06-8068-63f196883961","Type":"ContainerDied","Data":"6a5b2c370fb9ff622e3fe3277df22bd676c9eeeb71d397e0551445c21d34f6c5"}
Sep 19 11:39:36.344505 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:36.344462    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-tzkmq"]
Sep 19 11:39:37.036090 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.036067    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:39:37.079046 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.079018    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-cacerts\") pod \"f04747b6-1efd-4d06-8068-63f196883961\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") "
Sep 19 11:39:37.079213 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.079100    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f04747b6-1efd-4d06-8068-63f196883961-istio-csr-ca-configmap\") pod \"f04747b6-1efd-4d06-8068-63f196883961\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") "
Sep 19 11:39:37.079213 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.079123    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f04747b6-1efd-4d06-8068-63f196883961-local-certs\") pod \"f04747b6-1efd-4d06-8068-63f196883961\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") "
Sep 19 11:39:37.079213 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.079162    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-istio-kubeconfig\") pod \"f04747b6-1efd-4d06-8068-63f196883961\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") "
Sep 19 11:39:37.079213 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.079179    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f04747b6-1efd-4d06-8068-63f196883961-istio-token\") pod \"f04747b6-1efd-4d06-8068-63f196883961\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") "
Sep 19 11:39:37.079436 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.079285    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmgh9\" (UniqueName: \"kubernetes.io/projected/f04747b6-1efd-4d06-8068-63f196883961-kube-api-access-wmgh9\") pod \"f04747b6-1efd-4d06-8068-63f196883961\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") "
Sep 19 11:39:37.079580 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.079547    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f04747b6-1efd-4d06-8068-63f196883961-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "f04747b6-1efd-4d06-8068-63f196883961" (UID: "f04747b6-1efd-4d06-8068-63f196883961"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:39:37.079672 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.079318    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-istio-csr-dns-cert\") pod \"f04747b6-1efd-4d06-8068-63f196883961\" (UID: \"f04747b6-1efd-4d06-8068-63f196883961\") "
Sep 19 11:39:37.080124 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.079926    2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f04747b6-1efd-4d06-8068-63f196883961-istio-csr-ca-configmap\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:39:37.082526 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.082375    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-cacerts" (OuterVolumeSpecName: "cacerts") pod "f04747b6-1efd-4d06-8068-63f196883961" (UID: "f04747b6-1efd-4d06-8068-63f196883961"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:39:37.082526 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.082489    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f04747b6-1efd-4d06-8068-63f196883961-kube-api-access-wmgh9" (OuterVolumeSpecName: "kube-api-access-wmgh9") pod "f04747b6-1efd-4d06-8068-63f196883961" (UID: "f04747b6-1efd-4d06-8068-63f196883961"). InnerVolumeSpecName "kube-api-access-wmgh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:39:37.082678 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.082549    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f04747b6-1efd-4d06-8068-63f196883961-local-certs" (OuterVolumeSpecName: "local-certs") pod "f04747b6-1efd-4d06-8068-63f196883961" (UID: "f04747b6-1efd-4d06-8068-63f196883961"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:39:37.082678 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.082593    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f04747b6-1efd-4d06-8068-63f196883961-istio-token" (OuterVolumeSpecName: "istio-token") pod "f04747b6-1efd-4d06-8068-63f196883961" (UID: "f04747b6-1efd-4d06-8068-63f196883961"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:39:37.082788 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.082767    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "f04747b6-1efd-4d06-8068-63f196883961" (UID: "f04747b6-1efd-4d06-8068-63f196883961"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:39:37.083085 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.083063    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "f04747b6-1efd-4d06-8068-63f196883961" (UID: "f04747b6-1efd-4d06-8068-63f196883961"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:39:37.180735 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.180698    2559 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f04747b6-1efd-4d06-8068-63f196883961-local-certs\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:39:37.180735 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.180735    2559 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-istio-kubeconfig\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:39:37.180735 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.180744    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f04747b6-1efd-4d06-8068-63f196883961-istio-token\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:39:37.180968 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.180753    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmgh9\" (UniqueName: \"kubernetes.io/projected/f04747b6-1efd-4d06-8068-63f196883961-kube-api-access-wmgh9\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:39:37.180968 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.180767    2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-istio-csr-dns-cert\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:39:37.180968 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.180780    2559 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f04747b6-1efd-4d06-8068-63f196883961-cacerts\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:39:37.220949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.220924    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"
Sep 19 11:39:37.220949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.220940    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb" event={"ID":"f04747b6-1efd-4d06-8068-63f196883961","Type":"ContainerDied","Data":"d48125742ee66102a9c47f2e2b0328762a46148813997f2bdae31f16fee0ae10"}
Sep 19 11:39:37.221166 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.221005    2559 scope.go:117] "RemoveContainer" containerID="6a5b2c370fb9ff622e3fe3277df22bd676c9eeeb71d397e0551445c21d34f6c5"
Sep 19 11:39:37.244914 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.244887    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"]
Sep 19 11:39:37.252250 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:37.252226    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb"]
Sep 19 11:39:38.228204 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:38.228157    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv" event={"ID":"3d1f84f0-c532-4981-aa15-dbed60d05e18","Type":"ContainerStarted","Data":"6d1a10650a907a3c08cee8966927c9d916c93b4541a06c3ed2a2dbb1e6ad47c0"}
Sep 19 11:39:38.229884 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:38.229858    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-tzkmq" event={"ID":"de84840e-a833-4203-bdef-766036d68ba6","Type":"ContainerStarted","Data":"91b687c7baee7c86f27757e4f564766af9867f5049e842655602f7c829e73536"}
Sep 19 11:39:38.230030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:38.229918    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-tzkmq" podUID="de84840e-a833-4203-bdef-766036d68ba6" containerName="authorino" containerID="cri-o://91b687c7baee7c86f27757e4f564766af9867f5049e842655602f7c829e73536" gracePeriod=30
Sep 19 11:39:38.232637 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:38.232604    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr" event={"ID":"32f38755-5a54-4a59-a523-e8958e46b40f","Type":"ContainerStarted","Data":"064a8dea7dc75758bb559959a3a74710351d3520de149cd0a793f109e0779582"}
Sep 19 11:39:38.232763 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:38.232728    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr"
Sep 19 11:39:38.244814 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:38.244775    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-thdmv" podStartSLOduration=2.235892693 podStartE2EDuration="8.2447606s" totalImagesPullingTime="6.008867907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:39:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:39:38.242826872 +0000 UTC m=+330.165139752" watchObservedRunningTime="2026-09-19 11:39:38.2447606 +0000 UTC m=+330.167073480"
Sep 19 11:39:38.264343 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:38.264284    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-tzkmq" podStartSLOduration=1.655921862 podStartE2EDuration="7.264267357s" totalImagesPullingTime="5.608345495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:39:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:39:38.263935226 +0000 UTC m=+330.186248107" watchObservedRunningTime="2026-09-19 11:39:38.264267357 +0000 UTC m=+330.186580239"
Sep 19 11:39:38.279915 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:38.279866    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr" podStartSLOduration=2.036378573 podStartE2EDuration="7.279847973s" totalImagesPullingTime="5.2434694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:39:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:39:38.2785141 +0000 UTC m=+330.200826982" watchObservedRunningTime="2026-09-19 11:39:38.279847973 +0000 UTC m=+330.202160853"
Sep 19 11:39:38.480583 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:38.480530    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-tzkmq"
Sep 19 11:39:38.593525 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:38.593493    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27p67\" (UniqueName: \"kubernetes.io/projected/de84840e-a833-4203-bdef-766036d68ba6-kube-api-access-27p67\") pod \"de84840e-a833-4203-bdef-766036d68ba6\" (UID: \"de84840e-a833-4203-bdef-766036d68ba6\") "
Sep 19 11:39:38.594610 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:38.594589    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f04747b6-1efd-4d06-8068-63f196883961" path="/var/lib/kubelet/pods/f04747b6-1efd-4d06-8068-63f196883961/volumes"
Sep 19 11:39:38.595876 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:38.595846    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de84840e-a833-4203-bdef-766036d68ba6-kube-api-access-27p67" (OuterVolumeSpecName: "kube-api-access-27p67") pod "de84840e-a833-4203-bdef-766036d68ba6" (UID: "de84840e-a833-4203-bdef-766036d68ba6"). InnerVolumeSpecName "kube-api-access-27p67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:39:38.694563 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:38.694539    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27p67\" (UniqueName: \"kubernetes.io/projected/de84840e-a833-4203-bdef-766036d68ba6-kube-api-access-27p67\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:39:39.238721 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:39.238685    2559 generic.go:356] "Generic (PLEG): container finished" podID="de84840e-a833-4203-bdef-766036d68ba6" containerID="91b687c7baee7c86f27757e4f564766af9867f5049e842655602f7c829e73536" exitCode=0
Sep 19 11:39:39.238721 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:39.238725    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-tzkmq"
Sep 19 11:39:39.239167 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:39.238777    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-tzkmq" event={"ID":"de84840e-a833-4203-bdef-766036d68ba6","Type":"ContainerDied","Data":"91b687c7baee7c86f27757e4f564766af9867f5049e842655602f7c829e73536"}
Sep 19 11:39:39.239167 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:39.238811    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-tzkmq" event={"ID":"de84840e-a833-4203-bdef-766036d68ba6","Type":"ContainerDied","Data":"692619d8d15fdc121dda038b4a01c68371847f9ba528c5776356bfe1a7515a58"}
Sep 19 11:39:39.239167 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:39.238828    2559 scope.go:117] "RemoveContainer" containerID="91b687c7baee7c86f27757e4f564766af9867f5049e842655602f7c829e73536"
Sep 19 11:39:39.247999 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:39.247983    2559 scope.go:117] "RemoveContainer" containerID="91b687c7baee7c86f27757e4f564766af9867f5049e842655602f7c829e73536"
Sep 19 11:39:39.248250 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:39:39.248231    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91b687c7baee7c86f27757e4f564766af9867f5049e842655602f7c829e73536\": container with ID starting with 91b687c7baee7c86f27757e4f564766af9867f5049e842655602f7c829e73536 not found: ID does not exist" containerID="91b687c7baee7c86f27757e4f564766af9867f5049e842655602f7c829e73536"
Sep 19 11:39:39.248292 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:39.248258    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91b687c7baee7c86f27757e4f564766af9867f5049e842655602f7c829e73536"} err="failed to get container status \"91b687c7baee7c86f27757e4f564766af9867f5049e842655602f7c829e73536\": rpc error: code = NotFound desc = could not find container \"91b687c7baee7c86f27757e4f564766af9867f5049e842655602f7c829e73536\": container with ID starting with 91b687c7baee7c86f27757e4f564766af9867f5049e842655602f7c829e73536 not found: ID does not exist"
Sep 19 11:39:39.273477 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:39.273445    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-tzkmq"]
Sep 19 11:39:39.287144 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:39.287120    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-tzkmq"]
Sep 19 11:39:40.595136 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.595106    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de84840e-a833-4203-bdef-766036d68ba6" path="/var/lib/kubelet/pods/de84840e-a833-4203-bdef-766036d68ba6/volumes"
Sep 19 11:39:40.673615 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.673582    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"]
Sep 19 11:39:40.673922 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.673910    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f04747b6-1efd-4d06-8068-63f196883961" containerName="discovery"
Sep 19 11:39:40.673965 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.673924    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f04747b6-1efd-4d06-8068-63f196883961" containerName="discovery"
Sep 19 11:39:40.673965 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.673947    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de84840e-a833-4203-bdef-766036d68ba6" containerName="authorino"
Sep 19 11:39:40.673965 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.673952    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="de84840e-a833-4203-bdef-766036d68ba6" containerName="authorino"
Sep 19 11:39:40.674050 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.674015    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f04747b6-1efd-4d06-8068-63f196883961" containerName="discovery"
Sep 19 11:39:40.674050 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.674024    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="de84840e-a833-4203-bdef-766036d68ba6" containerName="authorino"
Sep 19 11:39:40.678335 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.678294    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:40.681305 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.681282    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Sep 19 11:39:40.681305 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.681297    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Sep 19 11:39:40.681502 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.681417    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Sep 19 11:39:40.681589 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.681561    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-87sgr\""
Sep 19 11:39:40.681717 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.681646    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Sep 19 11:39:40.681717 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.681697    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Sep 19 11:39:40.685617 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.685594    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"]
Sep 19 11:39:40.708807 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.708785    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e82cd179-7d1c-4d54-bb4b-5fe98b650efc-manager-config\") pod \"jobset-controller-manager-686b576b67-xwr22\" (UID: \"e82cd179-7d1c-4d54-bb4b-5fe98b650efc\") " pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:40.708910 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.708849    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w4pk\" (UniqueName: \"kubernetes.io/projected/e82cd179-7d1c-4d54-bb4b-5fe98b650efc-kube-api-access-7w4pk\") pod \"jobset-controller-manager-686b576b67-xwr22\" (UID: \"e82cd179-7d1c-4d54-bb4b-5fe98b650efc\") " pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:40.708910 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.708901    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e82cd179-7d1c-4d54-bb4b-5fe98b650efc-metrics-certs\") pod \"jobset-controller-manager-686b576b67-xwr22\" (UID: \"e82cd179-7d1c-4d54-bb4b-5fe98b650efc\") " pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:40.709004 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.708929    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e82cd179-7d1c-4d54-bb4b-5fe98b650efc-cert\") pod \"jobset-controller-manager-686b576b67-xwr22\" (UID: \"e82cd179-7d1c-4d54-bb4b-5fe98b650efc\") " pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:40.810372 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.810339    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e82cd179-7d1c-4d54-bb4b-5fe98b650efc-manager-config\") pod \"jobset-controller-manager-686b576b67-xwr22\" (UID: \"e82cd179-7d1c-4d54-bb4b-5fe98b650efc\") " pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:40.810539 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.810410    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w4pk\" (UniqueName: \"kubernetes.io/projected/e82cd179-7d1c-4d54-bb4b-5fe98b650efc-kube-api-access-7w4pk\") pod \"jobset-controller-manager-686b576b67-xwr22\" (UID: \"e82cd179-7d1c-4d54-bb4b-5fe98b650efc\") " pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:40.810539 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.810451    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e82cd179-7d1c-4d54-bb4b-5fe98b650efc-metrics-certs\") pod \"jobset-controller-manager-686b576b67-xwr22\" (UID: \"e82cd179-7d1c-4d54-bb4b-5fe98b650efc\") " pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:40.810539 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.810478    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e82cd179-7d1c-4d54-bb4b-5fe98b650efc-cert\") pod \"jobset-controller-manager-686b576b67-xwr22\" (UID: \"e82cd179-7d1c-4d54-bb4b-5fe98b650efc\") " pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:40.811235 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.811158    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e82cd179-7d1c-4d54-bb4b-5fe98b650efc-manager-config\") pod \"jobset-controller-manager-686b576b67-xwr22\" (UID: \"e82cd179-7d1c-4d54-bb4b-5fe98b650efc\") " pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:40.813766 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.813735    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e82cd179-7d1c-4d54-bb4b-5fe98b650efc-cert\") pod \"jobset-controller-manager-686b576b67-xwr22\" (UID: \"e82cd179-7d1c-4d54-bb4b-5fe98b650efc\") " pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:40.813888 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.813735    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e82cd179-7d1c-4d54-bb4b-5fe98b650efc-metrics-certs\") pod \"jobset-controller-manager-686b576b67-xwr22\" (UID: \"e82cd179-7d1c-4d54-bb4b-5fe98b650efc\") " pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:40.822687 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.822668    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w4pk\" (UniqueName: \"kubernetes.io/projected/e82cd179-7d1c-4d54-bb4b-5fe98b650efc-kube-api-access-7w4pk\") pod \"jobset-controller-manager-686b576b67-xwr22\" (UID: \"e82cd179-7d1c-4d54-bb4b-5fe98b650efc\") " pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:40.990271 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:40.990247    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:41.262490 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:41.262469    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"]
Sep 19 11:39:41.264580 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:39:41.264552    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode82cd179_7d1c_4d54_bb4b_5fe98b650efc.slice/crio-0002bed6a49b07692102acb55da169a32c117a43390bf29d693da322f9dd799e WatchSource:0}: Error finding container 0002bed6a49b07692102acb55da169a32c117a43390bf29d693da322f9dd799e: Status 404 returned error can't find the container with id 0002bed6a49b07692102acb55da169a32c117a43390bf29d693da322f9dd799e
Sep 19 11:39:41.677925 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:41.677839    2559 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-mmlpb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.24:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 19 11:39:41.677925 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:41.677890    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-mmlpb" podUID="f04747b6-1efd-4d06-8068-63f196883961" containerName="discovery" probeResult="failure" output="Get \"http://10.134.0.24:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 19 11:39:42.257750 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:42.257705    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22" event={"ID":"e82cd179-7d1c-4d54-bb4b-5fe98b650efc","Type":"ContainerStarted","Data":"0002bed6a49b07692102acb55da169a32c117a43390bf29d693da322f9dd799e"}
Sep 19 11:39:43.194160 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:43.194127    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-tpt4s"
Sep 19 11:39:44.269216 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:44.269169    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22" event={"ID":"e82cd179-7d1c-4d54-bb4b-5fe98b650efc","Type":"ContainerStarted","Data":"1f9b15f3935e41a486113ce67d6f49624f6d0b0ab47248441c9a7ecbc080ac62"}
Sep 19 11:39:44.269605 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:44.269271    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:44.283056 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:44.283003    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22" podStartSLOduration=1.6990460779999998 podStartE2EDuration="4.282984423s" totalImagesPullingTime="2.583938345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:39:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:39:44.28258232 +0000 UTC m=+336.204895191" watchObservedRunningTime="2026-09-19 11:39:44.282984423 +0000 UTC m=+336.205297305"
Sep 19 11:39:44.306024 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:44.305995    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tpt4s"]
Sep 19 11:39:44.306178 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:44.306158    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-tpt4s" podUID="ea50d270-9b5e-4bbc-ab20-d22db3ba9d78" containerName="limitador" containerID="cri-o://7c79448153cc16f4fce41b7ff719445a28e95c9f5f92163c1f0cda4c7080977b" gracePeriod=30
Sep 19 11:39:44.845487 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:44.845459    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tpt4s"
Sep 19 11:39:44.950341 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:44.950310    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w62kv\" (UniqueName: \"kubernetes.io/projected/ea50d270-9b5e-4bbc-ab20-d22db3ba9d78-kube-api-access-w62kv\") pod \"ea50d270-9b5e-4bbc-ab20-d22db3ba9d78\" (UID: \"ea50d270-9b5e-4bbc-ab20-d22db3ba9d78\") "
Sep 19 11:39:44.950495 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:44.950426    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ea50d270-9b5e-4bbc-ab20-d22db3ba9d78-config-file\") pod \"ea50d270-9b5e-4bbc-ab20-d22db3ba9d78\" (UID: \"ea50d270-9b5e-4bbc-ab20-d22db3ba9d78\") "
Sep 19 11:39:44.950741 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:44.950721    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea50d270-9b5e-4bbc-ab20-d22db3ba9d78-config-file" (OuterVolumeSpecName: "config-file") pod "ea50d270-9b5e-4bbc-ab20-d22db3ba9d78" (UID: "ea50d270-9b5e-4bbc-ab20-d22db3ba9d78"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:39:44.952480 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:44.952461    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea50d270-9b5e-4bbc-ab20-d22db3ba9d78-kube-api-access-w62kv" (OuterVolumeSpecName: "kube-api-access-w62kv") pod "ea50d270-9b5e-4bbc-ab20-d22db3ba9d78" (UID: "ea50d270-9b5e-4bbc-ab20-d22db3ba9d78"). InnerVolumeSpecName "kube-api-access-w62kv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:39:45.051004 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:45.050976    2559 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ea50d270-9b5e-4bbc-ab20-d22db3ba9d78-config-file\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:39:45.051004 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:45.050999    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w62kv\" (UniqueName: \"kubernetes.io/projected/ea50d270-9b5e-4bbc-ab20-d22db3ba9d78-kube-api-access-w62kv\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:39:45.274983 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:45.274906    2559 generic.go:356] "Generic (PLEG): container finished" podID="ea50d270-9b5e-4bbc-ab20-d22db3ba9d78" containerID="7c79448153cc16f4fce41b7ff719445a28e95c9f5f92163c1f0cda4c7080977b" exitCode=0
Sep 19 11:39:45.274983 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:45.274946    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tpt4s"
Sep 19 11:39:45.275426 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:45.274983    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tpt4s" event={"ID":"ea50d270-9b5e-4bbc-ab20-d22db3ba9d78","Type":"ContainerDied","Data":"7c79448153cc16f4fce41b7ff719445a28e95c9f5f92163c1f0cda4c7080977b"}
Sep 19 11:39:45.275426 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:45.275031    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tpt4s" event={"ID":"ea50d270-9b5e-4bbc-ab20-d22db3ba9d78","Type":"ContainerDied","Data":"62d9b4b82502cbaced1e8abfad456315c3afb72bcdc4ac4f8d314627ea61e23d"}
Sep 19 11:39:45.275426 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:45.275052    2559 scope.go:117] "RemoveContainer" containerID="7c79448153cc16f4fce41b7ff719445a28e95c9f5f92163c1f0cda4c7080977b"
Sep 19 11:39:45.284506 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:45.284493    2559 scope.go:117] "RemoveContainer" containerID="7c79448153cc16f4fce41b7ff719445a28e95c9f5f92163c1f0cda4c7080977b"
Sep 19 11:39:45.284794 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:39:45.284777    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c79448153cc16f4fce41b7ff719445a28e95c9f5f92163c1f0cda4c7080977b\": container with ID starting with 7c79448153cc16f4fce41b7ff719445a28e95c9f5f92163c1f0cda4c7080977b not found: ID does not exist" containerID="7c79448153cc16f4fce41b7ff719445a28e95c9f5f92163c1f0cda4c7080977b"
Sep 19 11:39:45.284833 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:45.284803    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c79448153cc16f4fce41b7ff719445a28e95c9f5f92163c1f0cda4c7080977b"} err="failed to get container status \"7c79448153cc16f4fce41b7ff719445a28e95c9f5f92163c1f0cda4c7080977b\": rpc error: code = NotFound desc = could not find container \"7c79448153cc16f4fce41b7ff719445a28e95c9f5f92163c1f0cda4c7080977b\": container with ID starting with 7c79448153cc16f4fce41b7ff719445a28e95c9f5f92163c1f0cda4c7080977b not found: ID does not exist"
Sep 19 11:39:45.297172 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:45.297152    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tpt4s"]
Sep 19 11:39:45.303001 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:45.302983    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tpt4s"]
Sep 19 11:39:46.028291 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.028259    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp"]
Sep 19 11:39:46.028726 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.028689    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea50d270-9b5e-4bbc-ab20-d22db3ba9d78" containerName="limitador"
Sep 19 11:39:46.028902 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.028889    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea50d270-9b5e-4bbc-ab20-d22db3ba9d78" containerName="limitador"
Sep 19 11:39:46.029119 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.029094    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea50d270-9b5e-4bbc-ab20-d22db3ba9d78" containerName="limitador"
Sep 19 11:39:46.033210 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.033179    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp"
Sep 19 11:39:46.037043 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.037020    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 11:39:46.037154 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.037048    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 19 11:39:46.037154 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.037084    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-bm5qw\""
Sep 19 11:39:46.037288 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.037175    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 11:39:46.042001 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.041979    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp"]
Sep 19 11:39:46.161253 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.161220    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/38cd5722-df7a-46c2-b49d-1875229db11c-data\") pod \"postgres-69fb4594cf-6l5wp\" (UID: \"38cd5722-df7a-46c2-b49d-1875229db11c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp"
Sep 19 11:39:46.161407 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.161265    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qtb4\" (UniqueName: \"kubernetes.io/projected/38cd5722-df7a-46c2-b49d-1875229db11c-kube-api-access-6qtb4\") pod \"postgres-69fb4594cf-6l5wp\" (UID: \"38cd5722-df7a-46c2-b49d-1875229db11c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp"
Sep 19 11:39:46.261852 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.261821    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/38cd5722-df7a-46c2-b49d-1875229db11c-data\") pod \"postgres-69fb4594cf-6l5wp\" (UID: \"38cd5722-df7a-46c2-b49d-1875229db11c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp"
Sep 19 11:39:46.262014 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.261872    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qtb4\" (UniqueName: \"kubernetes.io/projected/38cd5722-df7a-46c2-b49d-1875229db11c-kube-api-access-6qtb4\") pod \"postgres-69fb4594cf-6l5wp\" (UID: \"38cd5722-df7a-46c2-b49d-1875229db11c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp"
Sep 19 11:39:46.262312 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.262291    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/38cd5722-df7a-46c2-b49d-1875229db11c-data\") pod \"postgres-69fb4594cf-6l5wp\" (UID: \"38cd5722-df7a-46c2-b49d-1875229db11c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp"
Sep 19 11:39:46.277804 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.277778    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qtb4\" (UniqueName: \"kubernetes.io/projected/38cd5722-df7a-46c2-b49d-1875229db11c-kube-api-access-6qtb4\") pod \"postgres-69fb4594cf-6l5wp\" (UID: \"38cd5722-df7a-46c2-b49d-1875229db11c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp"
Sep 19 11:39:46.345076 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.345011    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp"
Sep 19 11:39:46.595014 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.594980    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea50d270-9b5e-4bbc-ab20-d22db3ba9d78" path="/var/lib/kubelet/pods/ea50d270-9b5e-4bbc-ab20-d22db3ba9d78/volumes"
Sep 19 11:39:46.616804 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:46.616696    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp"]
Sep 19 11:39:46.620553 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:39:46.620524    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38cd5722_df7a_46c2_b49d_1875229db11c.slice/crio-9140ee42b729e699a51cb0e17d1695f29bd70d4a0914725108c33fa4e0c46d8d WatchSource:0}: Error finding container 9140ee42b729e699a51cb0e17d1695f29bd70d4a0914725108c33fa4e0c46d8d: Status 404 returned error can't find the container with id 9140ee42b729e699a51cb0e17d1695f29bd70d4a0914725108c33fa4e0c46d8d
Sep 19 11:39:47.286452 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:47.286419    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp" event={"ID":"38cd5722-df7a-46c2-b49d-1875229db11c","Type":"ContainerStarted","Data":"9140ee42b729e699a51cb0e17d1695f29bd70d4a0914725108c33fa4e0c46d8d"}
Sep 19 11:39:52.315917 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:52.315875    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp" event={"ID":"38cd5722-df7a-46c2-b49d-1875229db11c","Type":"ContainerStarted","Data":"37d0f3adf43a3db713d326fadfa5585c28247d5f48c09d83df6b47f3dbaebea6"}
Sep 19 11:39:52.316288 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:52.315988    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp"
Sep 19 11:39:52.332752 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:52.332698    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp" podStartSLOduration=1.486338911 podStartE2EDuration="6.332678252s" totalImagesPullingTime="4.846339341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:39:52.330033214 +0000 UTC m=+344.252346096" watchObservedRunningTime="2026-09-19 11:39:52.332678252 +0000 UTC m=+344.254991136"
Sep 19 11:39:55.280778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:55.280740    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-686b576b67-xwr22"
Sep 19 11:39:58.349778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:39:58.349753    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6l5wp"
Sep 19 11:40:09.241702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:09.241669    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-78f7d99c47-22zfr"
Sep 19 11:40:27.611063 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:27.611031    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b8cf4dd6d-tvq6s"]
Sep 19 11:40:38.656404 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.656374    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7"]
Sep 19 11:40:38.659939 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.659923    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7"
Sep 19 11:40:38.662616 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.662593    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wg8gs\""
Sep 19 11:40:38.662743 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.662620    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 19 11:40:38.672876 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.672857    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7"]
Sep 19 11:40:38.687118 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.687097    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-cf2vb"]
Sep 19 11:40:38.690610 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.690594    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-cf2vb"
Sep 19 11:40:38.704895 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.704863    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-cf2vb"]
Sep 19 11:40:38.713835 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.713810    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhfls\" (UniqueName: \"kubernetes.io/projected/aa236dc9-323d-4d69-af0e-31787c8c5b24-kube-api-access-jhfls\") pod \"payload-pre-processing-59fbc47bf8-s8xn7\" (UID: \"aa236dc9-323d-4d69-af0e-31787c8c5b24\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7"
Sep 19 11:40:38.713935 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.713873    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa236dc9-323d-4d69-af0e-31787c8c5b24-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-s8xn7\" (UID: \"aa236dc9-323d-4d69-af0e-31787c8c5b24\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7"
Sep 19 11:40:38.814597 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.814558    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhfls\" (UniqueName: \"kubernetes.io/projected/aa236dc9-323d-4d69-af0e-31787c8c5b24-kube-api-access-jhfls\") pod \"payload-pre-processing-59fbc47bf8-s8xn7\" (UID: \"aa236dc9-323d-4d69-af0e-31787c8c5b24\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7"
Sep 19 11:40:38.814757 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.814611    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6292959-58d7-4e70-9bd2-ddc765c83de0-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-cf2vb\" (UID: \"a6292959-58d7-4e70-9bd2-ddc765c83de0\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cf2vb"
Sep 19 11:40:38.814757 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.814644    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw95r\" (UniqueName: \"kubernetes.io/projected/a6292959-58d7-4e70-9bd2-ddc765c83de0-kube-api-access-vw95r\") pod \"payload-processing-5b54b7fbb7-cf2vb\" (UID: \"a6292959-58d7-4e70-9bd2-ddc765c83de0\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cf2vb"
Sep 19 11:40:38.814757 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.814750    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa236dc9-323d-4d69-af0e-31787c8c5b24-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-s8xn7\" (UID: \"aa236dc9-323d-4d69-af0e-31787c8c5b24\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7"
Sep 19 11:40:38.815476 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.815456    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa236dc9-323d-4d69-af0e-31787c8c5b24-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-s8xn7\" (UID: \"aa236dc9-323d-4d69-af0e-31787c8c5b24\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7"
Sep 19 11:40:38.821856 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.821826    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhfls\" (UniqueName: \"kubernetes.io/projected/aa236dc9-323d-4d69-af0e-31787c8c5b24-kube-api-access-jhfls\") pod \"payload-pre-processing-59fbc47bf8-s8xn7\" (UID: \"aa236dc9-323d-4d69-af0e-31787c8c5b24\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7"
Sep 19 11:40:38.915958 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.915871    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6292959-58d7-4e70-9bd2-ddc765c83de0-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-cf2vb\" (UID: \"a6292959-58d7-4e70-9bd2-ddc765c83de0\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cf2vb"
Sep 19 11:40:38.915958 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.915915    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw95r\" (UniqueName: \"kubernetes.io/projected/a6292959-58d7-4e70-9bd2-ddc765c83de0-kube-api-access-vw95r\") pod \"payload-processing-5b54b7fbb7-cf2vb\" (UID: \"a6292959-58d7-4e70-9bd2-ddc765c83de0\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cf2vb"
Sep 19 11:40:38.916618 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.916596    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6292959-58d7-4e70-9bd2-ddc765c83de0-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-cf2vb\" (UID: \"a6292959-58d7-4e70-9bd2-ddc765c83de0\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cf2vb"
Sep 19 11:40:38.923660 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.923626    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw95r\" (UniqueName: \"kubernetes.io/projected/a6292959-58d7-4e70-9bd2-ddc765c83de0-kube-api-access-vw95r\") pod \"payload-processing-5b54b7fbb7-cf2vb\" (UID: \"a6292959-58d7-4e70-9bd2-ddc765c83de0\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cf2vb"
Sep 19 11:40:38.969469 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:38.969434    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7"
Sep 19 11:40:39.000824 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:39.000796    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-cf2vb"
Sep 19 11:40:39.246006 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:39.245983    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7"]
Sep 19 11:40:39.248099 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:40:39.248073    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa236dc9_323d_4d69_af0e_31787c8c5b24.slice/crio-ecac2b538a600539244058d110cbae42055c7286eccee1117d6723d42213de3b WatchSource:0}: Error finding container ecac2b538a600539244058d110cbae42055c7286eccee1117d6723d42213de3b: Status 404 returned error can't find the container with id ecac2b538a600539244058d110cbae42055c7286eccee1117d6723d42213de3b
Sep 19 11:40:39.277153 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:39.277132    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-cf2vb"]
Sep 19 11:40:39.279592 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:40:39.279570    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6292959_58d7_4e70_9bd2_ddc765c83de0.slice/crio-b86a4c92286374c1707fc8ee2d5032cc7fb9b767b24f4c902fdd0529a5a7262b WatchSource:0}: Error finding container b86a4c92286374c1707fc8ee2d5032cc7fb9b767b24f4c902fdd0529a5a7262b: Status 404 returned error can't find the container with id b86a4c92286374c1707fc8ee2d5032cc7fb9b767b24f4c902fdd0529a5a7262b
Sep 19 11:40:39.546261 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:39.546167    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7" event={"ID":"aa236dc9-323d-4d69-af0e-31787c8c5b24","Type":"ContainerStarted","Data":"ecac2b538a600539244058d110cbae42055c7286eccee1117d6723d42213de3b"}
Sep 19 11:40:39.547326 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:39.547293    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-cf2vb" event={"ID":"a6292959-58d7-4e70-9bd2-ddc765c83de0","Type":"ContainerStarted","Data":"b86a4c92286374c1707fc8ee2d5032cc7fb9b767b24f4c902fdd0529a5a7262b"}
Sep 19 11:40:41.559753 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:41.559712    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-cf2vb" event={"ID":"a6292959-58d7-4e70-9bd2-ddc765c83de0","Type":"ContainerStarted","Data":"95158a832e4270f65bbc560f891fcad9892a167e60fdfb74ddbd3adca8832891"}
Sep 19 11:40:41.560228 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:41.559872    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-cf2vb"
Sep 19 11:40:41.561320 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:41.561295    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7" event={"ID":"aa236dc9-323d-4d69-af0e-31787c8c5b24","Type":"ContainerStarted","Data":"94f185957f92c73f0f78ad615ab68151abd8687dfc918ba11b2beb8ec2fc4947"}
Sep 19 11:40:41.561444 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:41.561429    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7"
Sep 19 11:40:41.576559 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:41.576508    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-cf2vb" podStartSLOduration=2.368963139 podStartE2EDuration="3.57649301s" totalImagesPullingTime="1.207529871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:40:41.575747842 +0000 UTC m=+393.498060725" watchObservedRunningTime="2026-09-19 11:40:41.57649301 +0000 UTC m=+393.498805893"
Sep 19 11:40:41.590067 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:41.590018    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7" podStartSLOduration=2.353631935 podStartE2EDuration="3.590002535s" totalImagesPullingTime="1.2363706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:40:41.588908176 +0000 UTC m=+393.511221070" watchObservedRunningTime="2026-09-19 11:40:41.590002535 +0000 UTC m=+393.512315418"
Sep 19 11:40:52.568950 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:52.568916    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-s8xn7"
Sep 19 11:40:52.569344 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:52.568986    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-cf2vb"
Sep 19 11:40:52.636666 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:52.636619    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b8cf4dd6d-tvq6s" podUID="96dccbf0-5feb-4523-b5f7-709f5673c581" containerName="console" containerID="cri-o://d11ae55a0b90e132767cd9cb14c82953a59d3b0a6400325a003367e4071926e8" gracePeriod=15
Sep 19 11:40:52.885046 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:52.885016    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:40:53.046707 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.046670    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-console-config\") pod \"96dccbf0-5feb-4523-b5f7-709f5673c581\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") "
Sep 19 11:40:53.046872 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.046726    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-oauth-serving-cert\") pod \"96dccbf0-5feb-4523-b5f7-709f5673c581\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") "
Sep 19 11:40:53.046872 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.046766    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96dccbf0-5feb-4523-b5f7-709f5673c581-console-serving-cert\") pod \"96dccbf0-5feb-4523-b5f7-709f5673c581\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") "
Sep 19 11:40:53.046872 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.046794    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2kwv\" (UniqueName: \"kubernetes.io/projected/96dccbf0-5feb-4523-b5f7-709f5673c581-kube-api-access-n2kwv\") pod \"96dccbf0-5feb-4523-b5f7-709f5673c581\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") "
Sep 19 11:40:53.047036 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.046922    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96dccbf0-5feb-4523-b5f7-709f5673c581-console-oauth-config\") pod \"96dccbf0-5feb-4523-b5f7-709f5673c581\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") "
Sep 19 11:40:53.047036 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.047006    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-service-ca\") pod \"96dccbf0-5feb-4523-b5f7-709f5673c581\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") "
Sep 19 11:40:53.047133 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.047054    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-trusted-ca-bundle\") pod \"96dccbf0-5feb-4523-b5f7-709f5673c581\" (UID: \"96dccbf0-5feb-4523-b5f7-709f5673c581\") "
Sep 19 11:40:53.047133 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.047084    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-console-config" (OuterVolumeSpecName: "console-config") pod "96dccbf0-5feb-4523-b5f7-709f5673c581" (UID: "96dccbf0-5feb-4523-b5f7-709f5673c581"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:40:53.047265 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.047132    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "96dccbf0-5feb-4523-b5f7-709f5673c581" (UID: "96dccbf0-5feb-4523-b5f7-709f5673c581"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:40:53.047409 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.047391    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-console-config\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:40:53.047472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.047420    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-oauth-serving-cert\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:40:53.047472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.047391    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-service-ca" (OuterVolumeSpecName: "service-ca") pod "96dccbf0-5feb-4523-b5f7-709f5673c581" (UID: "96dccbf0-5feb-4523-b5f7-709f5673c581"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:40:53.047591 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.047571    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "96dccbf0-5feb-4523-b5f7-709f5673c581" (UID: "96dccbf0-5feb-4523-b5f7-709f5673c581"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:40:53.049112 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.049091    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96dccbf0-5feb-4523-b5f7-709f5673c581-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "96dccbf0-5feb-4523-b5f7-709f5673c581" (UID: "96dccbf0-5feb-4523-b5f7-709f5673c581"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:40:53.049644 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.049621    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96dccbf0-5feb-4523-b5f7-709f5673c581-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "96dccbf0-5feb-4523-b5f7-709f5673c581" (UID: "96dccbf0-5feb-4523-b5f7-709f5673c581"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:40:53.049734 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.049646    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96dccbf0-5feb-4523-b5f7-709f5673c581-kube-api-access-n2kwv" (OuterVolumeSpecName: "kube-api-access-n2kwv") pod "96dccbf0-5feb-4523-b5f7-709f5673c581" (UID: "96dccbf0-5feb-4523-b5f7-709f5673c581"). InnerVolumeSpecName "kube-api-access-n2kwv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:40:53.148327 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.148252    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96dccbf0-5feb-4523-b5f7-709f5673c581-console-serving-cert\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:40:53.148327 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.148277    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2kwv\" (UniqueName: \"kubernetes.io/projected/96dccbf0-5feb-4523-b5f7-709f5673c581-kube-api-access-n2kwv\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:40:53.148327 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.148287    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96dccbf0-5feb-4523-b5f7-709f5673c581-console-oauth-config\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:40:53.148327 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.148295    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:40:53.148327 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.148303    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96dccbf0-5feb-4523-b5f7-709f5673c581-trusted-ca-bundle\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:40:53.622841 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.622811    2559 generic.go:356] "Generic (PLEG): container finished" podID="96dccbf0-5feb-4523-b5f7-709f5673c581" containerID="d11ae55a0b90e132767cd9cb14c82953a59d3b0a6400325a003367e4071926e8" exitCode=0
Sep 19 11:40:53.623331 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.622864    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8cf4dd6d-tvq6s"
Sep 19 11:40:53.623331 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.622910    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8cf4dd6d-tvq6s" event={"ID":"96dccbf0-5feb-4523-b5f7-709f5673c581","Type":"ContainerDied","Data":"d11ae55a0b90e132767cd9cb14c82953a59d3b0a6400325a003367e4071926e8"}
Sep 19 11:40:53.623331 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.622959    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8cf4dd6d-tvq6s" event={"ID":"96dccbf0-5feb-4523-b5f7-709f5673c581","Type":"ContainerDied","Data":"63bf5dcc3783bf3b1e1abdc4bee4bcf2c4bac6b8e85b09c988617a2d9fb20827"}
Sep 19 11:40:53.623331 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.622979    2559 scope.go:117] "RemoveContainer" containerID="d11ae55a0b90e132767cd9cb14c82953a59d3b0a6400325a003367e4071926e8"
Sep 19 11:40:53.632091 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.632076    2559 scope.go:117] "RemoveContainer" containerID="d11ae55a0b90e132767cd9cb14c82953a59d3b0a6400325a003367e4071926e8"
Sep 19 11:40:53.632368 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:40:53.632352    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d11ae55a0b90e132767cd9cb14c82953a59d3b0a6400325a003367e4071926e8\": container with ID starting with d11ae55a0b90e132767cd9cb14c82953a59d3b0a6400325a003367e4071926e8 not found: ID does not exist" containerID="d11ae55a0b90e132767cd9cb14c82953a59d3b0a6400325a003367e4071926e8"
Sep 19 11:40:53.632449 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.632374    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d11ae55a0b90e132767cd9cb14c82953a59d3b0a6400325a003367e4071926e8"} err="failed to get container status \"d11ae55a0b90e132767cd9cb14c82953a59d3b0a6400325a003367e4071926e8\": rpc error: code = NotFound desc = could not find container \"d11ae55a0b90e132767cd9cb14c82953a59d3b0a6400325a003367e4071926e8\": container with ID starting with d11ae55a0b90e132767cd9cb14c82953a59d3b0a6400325a003367e4071926e8 not found: ID does not exist"
Sep 19 11:40:53.643151 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.643125    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b8cf4dd6d-tvq6s"]
Sep 19 11:40:53.650212 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:53.646047    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b8cf4dd6d-tvq6s"]
Sep 19 11:40:54.595957 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:40:54.595924    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96dccbf0-5feb-4523-b5f7-709f5673c581" path="/var/lib/kubelet/pods/96dccbf0-5feb-4523-b5f7-709f5673c581/volumes"
Sep 19 11:41:24.137542 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.137504    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"]
Sep 19 11:41:24.138085 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.137871    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96dccbf0-5feb-4523-b5f7-709f5673c581" containerName="console"
Sep 19 11:41:24.138085 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.137881    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="96dccbf0-5feb-4523-b5f7-709f5673c581" containerName="console"
Sep 19 11:41:24.138085 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.137945    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="96dccbf0-5feb-4523-b5f7-709f5673c581" containerName="console"
Sep 19 11:41:24.143021 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.143003    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.145288 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.145268    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6gh2s\""
Sep 19 11:41:24.145393 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.145297    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 19 11:41:24.146161 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.146144    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 19 11:41:24.146283 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.146167    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 19 11:41:24.151546 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.151526    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"]
Sep 19 11:41:24.216758 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.216728    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-dshm\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.216915 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.216780    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-tmp-dir\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.216915 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.216806    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-tls-certs\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.216915 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.216857    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-model-cache\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.216915 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.216890    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-home\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.217070 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.216953    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l87x2\" (UniqueName: \"kubernetes.io/projected/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-kube-api-access-l87x2\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.318429 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.318396    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l87x2\" (UniqueName: \"kubernetes.io/projected/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-kube-api-access-l87x2\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.318608 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.318442    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-dshm\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.318608 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.318502    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-tmp-dir\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.318608 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.318522    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-tls-certs\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.318608 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.318555    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-model-cache\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.318608 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.318586    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-home\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.319037 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.319015    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-tmp-dir\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.319126 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.319048    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-model-cache\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.319126 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.319063    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-home\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.321048 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.321026    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-dshm\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.321259 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.321242    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-tls-certs\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.325644 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.325623    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l87x2\" (UniqueName: \"kubernetes.io/projected/6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642-kube-api-access-l87x2\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-kjh7j\" (UID: \"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.454571 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.454529    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:24.727379 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.727357    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"]
Sep 19 11:41:24.729016 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:41:24.728990    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d60fd1f_a2fd_485e_a8ad_d6e53a2b6642.slice/crio-67ab4b09842bfd2853e3824ef8d3c4b1458268f415ecb7cb9a8520e08790d94c WatchSource:0}: Error finding container 67ab4b09842bfd2853e3824ef8d3c4b1458268f415ecb7cb9a8520e08790d94c: Status 404 returned error can't find the container with id 67ab4b09842bfd2853e3824ef8d3c4b1458268f415ecb7cb9a8520e08790d94c
Sep 19 11:41:24.771867 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:24.771832    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j" event={"ID":"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642","Type":"ContainerStarted","Data":"67ab4b09842bfd2853e3824ef8d3c4b1458268f415ecb7cb9a8520e08790d94c"}
Sep 19 11:41:25.779391 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:25.779355    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j" event={"ID":"6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642","Type":"ContainerStarted","Data":"dd3734975d3f1dd785fb3a352a035adae623c5b5cb926a5f19c2b2fdf61f0adc"}
Sep 19 11:41:25.798731 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:25.798667    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j" podStartSLOduration=0.858162725 podStartE2EDuration="1.798647934s" totalImagesPullingTime="940.485209ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:41:25.796837276 +0000 UTC m=+437.719150158" watchObservedRunningTime="2026-09-19 11:41:25.798647934 +0000 UTC m=+437.720960817"
Sep 19 11:41:26.455010 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:26.454976    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:34.455161 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:34.455071    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:34.467832 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:34.467805    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:34.836872 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:34.836797    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-kjh7j"
Sep 19 11:41:44.425024 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.424981    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"]
Sep 19 11:41:44.430701 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.430678    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.432908 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.432891    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 19 11:41:44.436558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.436535    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"]
Sep 19 11:41:44.611858 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.611826    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhwfg\" (UniqueName: \"kubernetes.io/projected/96bbadaf-5cc8-499d-8754-835cb5eff26e-kube-api-access-rhwfg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.612007 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.611865    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96bbadaf-5cc8-499d-8754-835cb5eff26e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.612007 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.611926    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96bbadaf-5cc8-499d-8754-835cb5eff26e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.612007 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.611942    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96bbadaf-5cc8-499d-8754-835cb5eff26e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.612007 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.611960    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96bbadaf-5cc8-499d-8754-835cb5eff26e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.612177 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.612015    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96bbadaf-5cc8-499d-8754-835cb5eff26e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.712946 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.712910    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96bbadaf-5cc8-499d-8754-835cb5eff26e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.714316 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.714292    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96bbadaf-5cc8-499d-8754-835cb5eff26e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.714766 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.714751    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96bbadaf-5cc8-499d-8754-835cb5eff26e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.715157 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.715142    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96bbadaf-5cc8-499d-8754-835cb5eff26e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.715681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.715664    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96bbadaf-5cc8-499d-8754-835cb5eff26e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.716226 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.716206    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhwfg\" (UniqueName: \"kubernetes.io/projected/96bbadaf-5cc8-499d-8754-835cb5eff26e-kube-api-access-rhwfg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.716719 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.714702    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96bbadaf-5cc8-499d-8754-835cb5eff26e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.716847 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.713546    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96bbadaf-5cc8-499d-8754-835cb5eff26e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.716938 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.715088    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96bbadaf-5cc8-499d-8754-835cb5eff26e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.720153 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.720129    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96bbadaf-5cc8-499d-8754-835cb5eff26e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.720531 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.720510    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96bbadaf-5cc8-499d-8754-835cb5eff26e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.723830 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.723811    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhwfg\" (UniqueName: \"kubernetes.io/projected/96bbadaf-5cc8-499d-8754-835cb5eff26e-kube-api-access-rhwfg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v\" (UID: \"96bbadaf-5cc8-499d-8754-835cb5eff26e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:44.743478 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:44.743453    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:45.033918 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:45.033890    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"]
Sep 19 11:41:45.035877 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:41:45.035849    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96bbadaf_5cc8_499d_8754_835cb5eff26e.slice/crio-f85fb153e35b7942fe9925cc868316ff01aaf8dd200d64de417bd1a39b661410 WatchSource:0}: Error finding container f85fb153e35b7942fe9925cc868316ff01aaf8dd200d64de417bd1a39b661410: Status 404 returned error can't find the container with id f85fb153e35b7942fe9925cc868316ff01aaf8dd200d64de417bd1a39b661410
Sep 19 11:41:45.884046 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:45.884007    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v" event={"ID":"96bbadaf-5cc8-499d-8754-835cb5eff26e","Type":"ContainerStarted","Data":"6b30bf659d5a39f4d31ac66c27643f25dddb6c2ac36f8121f0f38d87ad10514b"}
Sep 19 11:41:45.884506 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:45.884101    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v" event={"ID":"96bbadaf-5cc8-499d-8754-835cb5eff26e","Type":"ContainerStarted","Data":"f85fb153e35b7942fe9925cc868316ff01aaf8dd200d64de417bd1a39b661410"}
Sep 19 11:41:45.904166 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:45.904115    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v" podStartSLOduration=1.730808293 podStartE2EDuration="1.90409986s" totalImagesPullingTime="173.291567ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:41:45.902353229 +0000 UTC m=+457.824666112" watchObservedRunningTime="2026-09-19 11:41:45.90409986 +0000 UTC m=+457.826412741"
Sep 19 11:41:46.744419 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:46.744388    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:54.743837 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:54.743802    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:54.756316 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:54.756290    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:41:54.937674 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:41:54.937651    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v"
Sep 19 11:43:39.206536 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.206499    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"]
Sep 19 11:43:39.211216 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.211169    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.213563 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.213542    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-7eb15e-openshift-default-dockercfg-2vkz9\""
Sep 19 11:43:39.223632 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.223613    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"]
Sep 19 11:43:39.299127 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.299098    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-workload-certs\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.299315 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.299137    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-token\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.299315 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.299248    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-credential-socket\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.299315 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.299292    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-data\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.299481 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.299328    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istiod-ca-cert\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.299481 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.299397    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-envoy\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.299481 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.299466    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh75b\" (UniqueName: \"kubernetes.io/projected/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-kube-api-access-gh75b\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.299617 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.299495    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-podinfo\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.299617 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.299516    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-workload-socket\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.400487 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.400450    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh75b\" (UniqueName: \"kubernetes.io/projected/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-kube-api-access-gh75b\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.400487 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.400493    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-podinfo\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.400680 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.400512    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-workload-socket\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.400680 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.400557    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-workload-certs\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.400680 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.400574    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-token\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.400680 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.400597    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-credential-socket\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.400884 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.400684    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-data\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.400884 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.400742    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istiod-ca-cert\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.400884 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.400820    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-envoy\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.401031 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.401012    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-workload-socket\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.401091 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.401057    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-credential-socket\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.401145 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.401099    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-workload-certs\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.401275 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.401209    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-data\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.401472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.401444    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istiod-ca-cert\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.403057 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.403038    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-podinfo\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.403145 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.403134    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-envoy\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.407837 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.407812    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh75b\" (UniqueName: \"kubernetes.io/projected/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-kube-api-access-gh75b\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.407943 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.407876    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-token\") pod \"e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") " pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.522433 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.522346    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:39.797518 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:39.797496    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"]
Sep 19 11:43:39.799770 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:43:39.799744    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8109c4b3_2b3e_4f95_bf02_b26cdcda973d.slice/crio-1d7b7709fd516dd1d1eeb0497813b81684a08b8025367077ec43c6bfe00359ed WatchSource:0}: Error finding container 1d7b7709fd516dd1d1eeb0497813b81684a08b8025367077ec43c6bfe00359ed: Status 404 returned error can't find the container with id 1d7b7709fd516dd1d1eeb0497813b81684a08b8025367077ec43c6bfe00359ed
Sep 19 11:43:40.446369 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:40.446333    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72" event={"ID":"8109c4b3-2b3e-4f95-bf02-b26cdcda973d","Type":"ContainerStarted","Data":"1d7b7709fd516dd1d1eeb0497813b81684a08b8025367077ec43c6bfe00359ed"}
Sep 19 11:43:42.386131 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:42.386092    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:43:42.386449 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:42.386168    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:43:42.386449 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:42.386210    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:43:42.460299 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:42.460268    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72" event={"ID":"8109c4b3-2b3e-4f95-bf02-b26cdcda973d","Type":"ContainerStarted","Data":"2f2165ed2580b706bb3e95a59912847fc59788f718ee05061d7c9f9f4eda516e"}
Sep 19 11:43:42.484982 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:42.484744    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72" podStartSLOduration=0.90127493 podStartE2EDuration="3.484722708s" totalImagesPullingTime="2.583447778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:43:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:43:42.482346087 +0000 UTC m=+574.404658969" watchObservedRunningTime="2026-09-19 11:43:42.484722708 +0000 UTC m=+574.407035589"
Sep 19 11:43:42.523243 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:42.523185    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:43.528057 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:43.528014    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:44.470837 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:44.470804    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:44.471718 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:44.471700    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:43:45.175211 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.175163    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"]
Sep 19 11:43:45.178714 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.178698    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"
Sep 19 11:43:45.181031 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.181009    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-333f49-dockercfg-t2pbw\""
Sep 19 11:43:45.181171 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.181110    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-333f49\""
Sep 19 11:43:45.187376 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.187354    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"]
Sep 19 11:43:45.256309 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.256281    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szwqh\" (UniqueName: \"kubernetes.io/projected/3ffbb330-4619-4940-85f8-576862e1b9d4-kube-api-access-szwqh\") pod \"payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt\" (UID: \"3ffbb330-4619-4940-85f8-576862e1b9d4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"
Sep 19 11:43:45.256456 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.256342    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ffbb330-4619-4940-85f8-576862e1b9d4-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt\" (UID: \"3ffbb330-4619-4940-85f8-576862e1b9d4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"
Sep 19 11:43:45.291592 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.291556    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"]
Sep 19 11:43:45.294997 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.294975    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"
Sep 19 11:43:45.307014 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.306990    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"]
Sep 19 11:43:45.357310 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.357280    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szwqh\" (UniqueName: \"kubernetes.io/projected/3ffbb330-4619-4940-85f8-576862e1b9d4-kube-api-access-szwqh\") pod \"payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt\" (UID: \"3ffbb330-4619-4940-85f8-576862e1b9d4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"
Sep 19 11:43:45.357487 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.357344    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10cacae5-7a0b-4f43-9002-d591f799e35c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms\" (UID: \"10cacae5-7a0b-4f43-9002-d591f799e35c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"
Sep 19 11:43:45.357487 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.357378    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ffbb330-4619-4940-85f8-576862e1b9d4-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt\" (UID: \"3ffbb330-4619-4940-85f8-576862e1b9d4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"
Sep 19 11:43:45.357487 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.357394    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pp42\" (UniqueName: \"kubernetes.io/projected/10cacae5-7a0b-4f43-9002-d591f799e35c-kube-api-access-5pp42\") pod \"payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms\" (UID: \"10cacae5-7a0b-4f43-9002-d591f799e35c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"
Sep 19 11:43:45.358229 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.358211    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ffbb330-4619-4940-85f8-576862e1b9d4-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt\" (UID: \"3ffbb330-4619-4940-85f8-576862e1b9d4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"
Sep 19 11:43:45.364805 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.364783    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szwqh\" (UniqueName: \"kubernetes.io/projected/3ffbb330-4619-4940-85f8-576862e1b9d4-kube-api-access-szwqh\") pod \"payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt\" (UID: \"3ffbb330-4619-4940-85f8-576862e1b9d4\") " pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"
Sep 19 11:43:45.458134 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.458099    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10cacae5-7a0b-4f43-9002-d591f799e35c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms\" (UID: \"10cacae5-7a0b-4f43-9002-d591f799e35c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"
Sep 19 11:43:45.458320 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.458149    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pp42\" (UniqueName: \"kubernetes.io/projected/10cacae5-7a0b-4f43-9002-d591f799e35c-kube-api-access-5pp42\") pod \"payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms\" (UID: \"10cacae5-7a0b-4f43-9002-d591f799e35c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"
Sep 19 11:43:45.458880 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.458860    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10cacae5-7a0b-4f43-9002-d591f799e35c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms\" (UID: \"10cacae5-7a0b-4f43-9002-d591f799e35c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"
Sep 19 11:43:45.466356 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.466333    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pp42\" (UniqueName: \"kubernetes.io/projected/10cacae5-7a0b-4f43-9002-d591f799e35c-kube-api-access-5pp42\") pod \"payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms\" (UID: \"10cacae5-7a0b-4f43-9002-d591f799e35c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"
Sep 19 11:43:45.489771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.489746    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"
Sep 19 11:43:45.606895 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.606862    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"
Sep 19 11:43:45.765268 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.765170    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"]
Sep 19 11:43:45.769154 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:43:45.769116    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ffbb330_4619_4940_85f8_576862e1b9d4.slice/crio-5e6f92e487d6650621e2e231c8a231abec571857feb10e1fba656bb988c599a0 WatchSource:0}: Error finding container 5e6f92e487d6650621e2e231c8a231abec571857feb10e1fba656bb988c599a0: Status 404 returned error can't find the container with id 5e6f92e487d6650621e2e231c8a231abec571857feb10e1fba656bb988c599a0
Sep 19 11:43:45.882397 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:45.882368    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"]
Sep 19 11:43:45.884344 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:43:45.884313    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10cacae5_7a0b_4f43_9002_d591f799e35c.slice/crio-3f5b597df8c5516c19a3691ac9e37184424e55ea0b615f150e6077f1b7e8a417 WatchSource:0}: Error finding container 3f5b597df8c5516c19a3691ac9e37184424e55ea0b615f150e6077f1b7e8a417: Status 404 returned error can't find the container with id 3f5b597df8c5516c19a3691ac9e37184424e55ea0b615f150e6077f1b7e8a417
Sep 19 11:43:46.484710 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:46.484673    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms" event={"ID":"10cacae5-7a0b-4f43-9002-d591f799e35c","Type":"ContainerStarted","Data":"699cdbe17c534c47ca38c77c631f322f25d1626279824fc13da9a39b490c1e7b"}
Sep 19 11:43:46.485234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:46.484773    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms" event={"ID":"10cacae5-7a0b-4f43-9002-d591f799e35c","Type":"ContainerStarted","Data":"3f5b597df8c5516c19a3691ac9e37184424e55ea0b615f150e6077f1b7e8a417"}
Sep 19 11:43:46.485234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:46.484806    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"
Sep 19 11:43:46.486802 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:46.486769    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt" event={"ID":"3ffbb330-4619-4940-85f8-576862e1b9d4","Type":"ContainerStarted","Data":"cce14ff69b22362f8933fa0dec21cb1086a442dbf155456b05a0630d0b13d575"}
Sep 19 11:43:46.486938 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:46.486823    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt" event={"ID":"3ffbb330-4619-4940-85f8-576862e1b9d4","Type":"ContainerStarted","Data":"5e6f92e487d6650621e2e231c8a231abec571857feb10e1fba656bb988c599a0"}
Sep 19 11:43:46.500882 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:46.500840    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms" podStartSLOduration=1.5008266749999999 podStartE2EDuration="1.500826675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:43:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:43:46.499082805 +0000 UTC m=+578.421395687" watchObservedRunningTime="2026-09-19 11:43:46.500826675 +0000 UTC m=+578.423139585"
Sep 19 11:43:46.514178 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:46.514132    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt" podStartSLOduration=1.514116838 podStartE2EDuration="1.514116838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:43:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:43:46.513882691 +0000 UTC m=+578.436195565" watchObservedRunningTime="2026-09-19 11:43:46.514116838 +0000 UTC m=+578.436429725"
Sep 19 11:43:47.144910 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.144876    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"]
Sep 19 11:43:47.148540 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.148523    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"
Sep 19 11:43:47.151017 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.150997    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-7eb15e-dockercfg-67rmk\""
Sep 19 11:43:47.151117 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.151003    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-7eb15e\""
Sep 19 11:43:47.158209 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.158166    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"]
Sep 19 11:43:47.177416 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.177387    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"]
Sep 19 11:43:47.177818 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.177785    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqcn9\" (UniqueName: \"kubernetes.io/projected/0c17815d-0860-4275-896b-02b6324cb3df-kube-api-access-xqcn9\") pod \"payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8\" (UID: \"0c17815d-0860-4275-896b-02b6324cb3df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"
Sep 19 11:43:47.177934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.177834    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c17815d-0860-4275-896b-02b6324cb3df-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8\" (UID: \"0c17815d-0860-4275-896b-02b6324cb3df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"
Sep 19 11:43:47.181074 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.181058    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"
Sep 19 11:43:47.191765 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.191739    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"]
Sep 19 11:43:47.278907 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.278876    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqcn9\" (UniqueName: \"kubernetes.io/projected/0c17815d-0860-4275-896b-02b6324cb3df-kube-api-access-xqcn9\") pod \"payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8\" (UID: \"0c17815d-0860-4275-896b-02b6324cb3df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"
Sep 19 11:43:47.278907 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.278918    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c17815d-0860-4275-896b-02b6324cb3df-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8\" (UID: \"0c17815d-0860-4275-896b-02b6324cb3df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"
Sep 19 11:43:47.279251 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.278967    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fqhp\" (UniqueName: \"kubernetes.io/projected/35937834-e7f4-48ab-87d2-f9ad269cf8c7-kube-api-access-4fqhp\") pod \"payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w\" (UID: \"35937834-e7f4-48ab-87d2-f9ad269cf8c7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"
Sep 19 11:43:47.279251 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.279001    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35937834-e7f4-48ab-87d2-f9ad269cf8c7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w\" (UID: \"35937834-e7f4-48ab-87d2-f9ad269cf8c7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"
Sep 19 11:43:47.279845 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.279820    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c17815d-0860-4275-896b-02b6324cb3df-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8\" (UID: \"0c17815d-0860-4275-896b-02b6324cb3df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"
Sep 19 11:43:47.287706 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.287683    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqcn9\" (UniqueName: \"kubernetes.io/projected/0c17815d-0860-4275-896b-02b6324cb3df-kube-api-access-xqcn9\") pod \"payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8\" (UID: \"0c17815d-0860-4275-896b-02b6324cb3df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"
Sep 19 11:43:47.380131 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.380082    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fqhp\" (UniqueName: \"kubernetes.io/projected/35937834-e7f4-48ab-87d2-f9ad269cf8c7-kube-api-access-4fqhp\") pod \"payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w\" (UID: \"35937834-e7f4-48ab-87d2-f9ad269cf8c7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"
Sep 19 11:43:47.380335 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.380146    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35937834-e7f4-48ab-87d2-f9ad269cf8c7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w\" (UID: \"35937834-e7f4-48ab-87d2-f9ad269cf8c7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"
Sep 19 11:43:47.380803 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.380784    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35937834-e7f4-48ab-87d2-f9ad269cf8c7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w\" (UID: \"35937834-e7f4-48ab-87d2-f9ad269cf8c7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"
Sep 19 11:43:47.388655 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.388627    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fqhp\" (UniqueName: \"kubernetes.io/projected/35937834-e7f4-48ab-87d2-f9ad269cf8c7-kube-api-access-4fqhp\") pod \"payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w\" (UID: \"35937834-e7f4-48ab-87d2-f9ad269cf8c7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"
Sep 19 11:43:47.461545 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.461520    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"
Sep 19 11:43:47.492214 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.492171    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"
Sep 19 11:43:47.493387 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.493363    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"
Sep 19 11:43:47.742875 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.742848    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"]
Sep 19 11:43:47.744030 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:43:47.743979    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c17815d_0860_4275_896b_02b6324cb3df.slice/crio-55e98077750940614054e0d56c7c2b3d75ac3c1de0e787458f1b75e8f85d3ab3 WatchSource:0}: Error finding container 55e98077750940614054e0d56c7c2b3d75ac3c1de0e787458f1b75e8f85d3ab3: Status 404 returned error can't find the container with id 55e98077750940614054e0d56c7c2b3d75ac3c1de0e787458f1b75e8f85d3ab3
Sep 19 11:43:47.785472 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:43:47.785419    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35937834_e7f4_48ab_87d2_f9ad269cf8c7.slice/crio-0e85f080559cdd9510ec711af24f728f29073375d24223e47f26e271a0e7ba77 WatchSource:0}: Error finding container 0e85f080559cdd9510ec711af24f728f29073375d24223e47f26e271a0e7ba77: Status 404 returned error can't find the container with id 0e85f080559cdd9510ec711af24f728f29073375d24223e47f26e271a0e7ba77
Sep 19 11:43:47.785590 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:47.785547    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"]
Sep 19 11:43:48.501147 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:48.501103    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8" event={"ID":"0c17815d-0860-4275-896b-02b6324cb3df","Type":"ContainerStarted","Data":"f55959b71cab5ebd7a0f2b743d3be921a2ba07e0f5b2c3132d935911b654fb5f"}
Sep 19 11:43:48.501612 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:48.501162    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8" event={"ID":"0c17815d-0860-4275-896b-02b6324cb3df","Type":"ContainerStarted","Data":"55e98077750940614054e0d56c7c2b3d75ac3c1de0e787458f1b75e8f85d3ab3"}
Sep 19 11:43:48.501612 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:48.501217    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"
Sep 19 11:43:48.502712 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:48.502687    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w" event={"ID":"35937834-e7f4-48ab-87d2-f9ad269cf8c7","Type":"ContainerStarted","Data":"c155f9440f9b2410bf7527c510a84ed436342e09217ef8a2c70d5c5ff02aca0e"}
Sep 19 11:43:48.502834 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:48.502736    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w" event={"ID":"35937834-e7f4-48ab-87d2-f9ad269cf8c7","Type":"ContainerStarted","Data":"0e85f080559cdd9510ec711af24f728f29073375d24223e47f26e271a0e7ba77"}
Sep 19 11:43:48.517638 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:48.517600    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8" podStartSLOduration=1.517584715 podStartE2EDuration="1.517584715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:43:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:43:48.515819723 +0000 UTC m=+580.438132605" watchObservedRunningTime="2026-09-19 11:43:48.517584715 +0000 UTC m=+580.439897596"
Sep 19 11:43:48.531890 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:48.531851    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w" podStartSLOduration=1.5318356990000002 podStartE2EDuration="1.531835699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:43:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:43:48.531344477 +0000 UTC m=+580.453657355" watchObservedRunningTime="2026-09-19 11:43:48.531835699 +0000 UTC m=+580.454148582"
Sep 19 11:43:49.249178 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.249142    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"]
Sep 19 11:43:49.252816 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.252799    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"
Sep 19 11:43:49.255315 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.255290    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-37daed-dockercfg-kqwjx\""
Sep 19 11:43:49.255436 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.255291    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-37daed\""
Sep 19 11:43:49.261835 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.261813    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"]
Sep 19 11:43:49.299026 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.298997    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ea9fc31-ab89-4382-b201-7d0640395506-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx\" (UID: \"8ea9fc31-ab89-4382-b201-7d0640395506\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"
Sep 19 11:43:49.299163 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.299039    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpgdn\" (UniqueName: \"kubernetes.io/projected/8ea9fc31-ab89-4382-b201-7d0640395506-kube-api-access-rpgdn\") pod \"payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx\" (UID: \"8ea9fc31-ab89-4382-b201-7d0640395506\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"
Sep 19 11:43:49.368816 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.368783    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"]
Sep 19 11:43:49.372328 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.372309    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"
Sep 19 11:43:49.380838 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.380487    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"]
Sep 19 11:43:49.399771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.399743    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ea9fc31-ab89-4382-b201-7d0640395506-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx\" (UID: \"8ea9fc31-ab89-4382-b201-7d0640395506\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"
Sep 19 11:43:49.399918 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.399796    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpgdn\" (UniqueName: \"kubernetes.io/projected/8ea9fc31-ab89-4382-b201-7d0640395506-kube-api-access-rpgdn\") pod \"payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx\" (UID: \"8ea9fc31-ab89-4382-b201-7d0640395506\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"
Sep 19 11:43:49.399918 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.399872    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9flgn\" (UniqueName: \"kubernetes.io/projected/9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9-kube-api-access-9flgn\") pod \"payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w\" (UID: \"9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"
Sep 19 11:43:49.400048 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.399924    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w\" (UID: \"9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"
Sep 19 11:43:49.400542 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.400520    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ea9fc31-ab89-4382-b201-7d0640395506-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx\" (UID: \"8ea9fc31-ab89-4382-b201-7d0640395506\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"
Sep 19 11:43:49.407406 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.407385    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpgdn\" (UniqueName: \"kubernetes.io/projected/8ea9fc31-ab89-4382-b201-7d0640395506-kube-api-access-rpgdn\") pod \"payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx\" (UID: \"8ea9fc31-ab89-4382-b201-7d0640395506\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"
Sep 19 11:43:49.500814 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.500741    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9flgn\" (UniqueName: \"kubernetes.io/projected/9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9-kube-api-access-9flgn\") pod \"payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w\" (UID: \"9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"
Sep 19 11:43:49.500936 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.500817    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w\" (UID: \"9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"
Sep 19 11:43:49.501555 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.501537    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w\" (UID: \"9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"
Sep 19 11:43:49.508517 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.508496    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"
Sep 19 11:43:49.508622 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.508548    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9flgn\" (UniqueName: \"kubernetes.io/projected/9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9-kube-api-access-9flgn\") pod \"payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w\" (UID: \"9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"
Sep 19 11:43:49.565127 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.565099    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"
Sep 19 11:43:49.685691 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.685055    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"
Sep 19 11:43:49.834904 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.834881    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"]
Sep 19 11:43:49.837107 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:43:49.837081    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ea9fc31_ab89_4382_b201_7d0640395506.slice/crio-60a226794e901f3b094711387627549b85340bb49460373de77652fb7a6020fc WatchSource:0}: Error finding container 60a226794e901f3b094711387627549b85340bb49460373de77652fb7a6020fc: Status 404 returned error can't find the container with id 60a226794e901f3b094711387627549b85340bb49460373de77652fb7a6020fc
Sep 19 11:43:49.963369 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:49.963347    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"]
Sep 19 11:43:49.965603 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:43:49.965576    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aa3c3e9_4096_49d0_8a5d_9c0a69ac78b9.slice/crio-8fa625b2bf758ee0f13da57be181cae17b555a9c50510c9cc38d237f93656336 WatchSource:0}: Error finding container 8fa625b2bf758ee0f13da57be181cae17b555a9c50510c9cc38d237f93656336: Status 404 returned error can't find the container with id 8fa625b2bf758ee0f13da57be181cae17b555a9c50510c9cc38d237f93656336
Sep 19 11:43:50.516085 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:50.516047    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx" event={"ID":"8ea9fc31-ab89-4382-b201-7d0640395506","Type":"ContainerStarted","Data":"32e5ebb2921c67607abeba62dc2ba7837baba9eaf713b60021df8cac1cf37b4f"}
Sep 19 11:43:50.516580 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:50.516294    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"
Sep 19 11:43:50.516580 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:50.516346    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx" event={"ID":"8ea9fc31-ab89-4382-b201-7d0640395506","Type":"ContainerStarted","Data":"60a226794e901f3b094711387627549b85340bb49460373de77652fb7a6020fc"}
Sep 19 11:43:50.517917 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:50.517896    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w" event={"ID":"9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9","Type":"ContainerStarted","Data":"0dc0fd588295d9a051215c0ca3a09b6586227b9216d427313458d6c4d2cb4185"}
Sep 19 11:43:50.518023 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:50.517934    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w" event={"ID":"9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9","Type":"ContainerStarted","Data":"8fa625b2bf758ee0f13da57be181cae17b555a9c50510c9cc38d237f93656336"}
Sep 19 11:43:50.518023 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:50.517956    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"
Sep 19 11:43:50.534245 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:50.534179    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx" podStartSLOduration=1.53416308 podStartE2EDuration="1.53416308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:43:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:43:50.53173371 +0000 UTC m=+582.454046593" watchObservedRunningTime="2026-09-19 11:43:50.53416308 +0000 UTC m=+582.456475961"
Sep 19 11:43:50.550083 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:50.550033    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w" podStartSLOduration=1.550018782 podStartE2EDuration="1.550018782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:43:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:43:50.548114044 +0000 UTC m=+582.470426927" watchObservedRunningTime="2026-09-19 11:43:50.550018782 +0000 UTC m=+582.472331662"
Sep 19 11:43:53.803649 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:53.803607    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"]
Sep 19 11:43:53.807605 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:53.807583    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"
Sep 19 11:43:53.810036 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:53.810013    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-7a524f\""
Sep 19 11:43:53.810366 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:53.810350    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-7a524f-dockercfg-jpnvd\""
Sep 19 11:43:53.817955 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:53.817934    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"]
Sep 19 11:43:53.944830 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:53.944793    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf44c\" (UniqueName: \"kubernetes.io/projected/0541c39e-ae1a-4231-8f83-b59f2dcff552-kube-api-access-mf44c\") pod \"payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll\" (UID: \"0541c39e-ae1a-4231-8f83-b59f2dcff552\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"
Sep 19 11:43:53.945013 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:53.944987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0541c39e-ae1a-4231-8f83-b59f2dcff552-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll\" (UID: \"0541c39e-ae1a-4231-8f83-b59f2dcff552\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"
Sep 19 11:43:54.045610 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:54.045573    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0541c39e-ae1a-4231-8f83-b59f2dcff552-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll\" (UID: \"0541c39e-ae1a-4231-8f83-b59f2dcff552\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"
Sep 19 11:43:54.045785 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:54.045630    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf44c\" (UniqueName: \"kubernetes.io/projected/0541c39e-ae1a-4231-8f83-b59f2dcff552-kube-api-access-mf44c\") pod \"payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll\" (UID: \"0541c39e-ae1a-4231-8f83-b59f2dcff552\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"
Sep 19 11:43:54.046254 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:54.046235    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0541c39e-ae1a-4231-8f83-b59f2dcff552-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll\" (UID: \"0541c39e-ae1a-4231-8f83-b59f2dcff552\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"
Sep 19 11:43:54.053543 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:54.053521    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf44c\" (UniqueName: \"kubernetes.io/projected/0541c39e-ae1a-4231-8f83-b59f2dcff552-kube-api-access-mf44c\") pod \"payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll\" (UID: \"0541c39e-ae1a-4231-8f83-b59f2dcff552\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"
Sep 19 11:43:54.119747 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:54.119690    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"
Sep 19 11:43:54.389789 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:54.389764    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"]
Sep 19 11:43:54.390579 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:43:54.390552    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0541c39e_ae1a_4231_8f83_b59f2dcff552.slice/crio-d7e4eb647e2f7c77b409254d2ffb026f26a6298ad59d1076fc4dc1a8dfce8247 WatchSource:0}: Error finding container d7e4eb647e2f7c77b409254d2ffb026f26a6298ad59d1076fc4dc1a8dfce8247: Status 404 returned error can't find the container with id d7e4eb647e2f7c77b409254d2ffb026f26a6298ad59d1076fc4dc1a8dfce8247
Sep 19 11:43:54.546534 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:54.546500    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll" event={"ID":"0541c39e-ae1a-4231-8f83-b59f2dcff552","Type":"ContainerStarted","Data":"969cd528ccb687178558ad2f2564be1b674181ed0f3d54e51200da601bde0d99"}
Sep 19 11:43:54.546721 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:54.546588    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll" event={"ID":"0541c39e-ae1a-4231-8f83-b59f2dcff552","Type":"ContainerStarted","Data":"d7e4eb647e2f7c77b409254d2ffb026f26a6298ad59d1076fc4dc1a8dfce8247"}
Sep 19 11:43:54.546721 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:54.546616    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"
Sep 19 11:43:54.562453 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:54.562408    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll" podStartSLOduration=1.562389909 podStartE2EDuration="1.562389909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:43:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:43:54.561122201 +0000 UTC m=+586.483435083" watchObservedRunningTime="2026-09-19 11:43:54.562389909 +0000 UTC m=+586.484702791"
Sep 19 11:43:55.678061 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:55.678029    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"]
Sep 19 11:43:55.681656 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:55.681629    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"
Sep 19 11:43:55.684069 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:55.684043    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-cb615d6f\""
Sep 19 11:43:55.684224 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:55.684045    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-cb615d6f-dockercfg-8x7fx\""
Sep 19 11:43:55.693340 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:55.693317    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"]
Sep 19 11:43:55.762075 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:55.762042    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d152572a-a22d-4ccd-bb4e-8a8b02030e35-plugins-config-volume\") pod \"payload-processing-e2e-ait-cb615d6f-855999c645-rr62m\" (UID: \"d152572a-a22d-4ccd-bb4e-8a8b02030e35\") " pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"
Sep 19 11:43:55.762254 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:55.762160    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24d6f\" (UniqueName: \"kubernetes.io/projected/d152572a-a22d-4ccd-bb4e-8a8b02030e35-kube-api-access-24d6f\") pod \"payload-processing-e2e-ait-cb615d6f-855999c645-rr62m\" (UID: \"d152572a-a22d-4ccd-bb4e-8a8b02030e35\") " pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"
Sep 19 11:43:55.863089 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:55.863058    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d152572a-a22d-4ccd-bb4e-8a8b02030e35-plugins-config-volume\") pod \"payload-processing-e2e-ait-cb615d6f-855999c645-rr62m\" (UID: \"d152572a-a22d-4ccd-bb4e-8a8b02030e35\") " pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"
Sep 19 11:43:55.863299 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:55.863147    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24d6f\" (UniqueName: \"kubernetes.io/projected/d152572a-a22d-4ccd-bb4e-8a8b02030e35-kube-api-access-24d6f\") pod \"payload-processing-e2e-ait-cb615d6f-855999c645-rr62m\" (UID: \"d152572a-a22d-4ccd-bb4e-8a8b02030e35\") " pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"
Sep 19 11:43:55.863828 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:55.863807    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d152572a-a22d-4ccd-bb4e-8a8b02030e35-plugins-config-volume\") pod \"payload-processing-e2e-ait-cb615d6f-855999c645-rr62m\" (UID: \"d152572a-a22d-4ccd-bb4e-8a8b02030e35\") " pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"
Sep 19 11:43:55.870631 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:55.870606    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24d6f\" (UniqueName: \"kubernetes.io/projected/d152572a-a22d-4ccd-bb4e-8a8b02030e35-kube-api-access-24d6f\") pod \"payload-processing-e2e-ait-cb615d6f-855999c645-rr62m\" (UID: \"d152572a-a22d-4ccd-bb4e-8a8b02030e35\") " pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"
Sep 19 11:43:55.995993 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:55.995955    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"
Sep 19 11:43:56.270915 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:56.270890    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"]
Sep 19 11:43:56.272644 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:43:56.272615    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd152572a_a22d_4ccd_bb4e_8a8b02030e35.slice/crio-d3a4334b2ef21e15529a99f7871fe92f0d39d3bb0346c08554cc50e7629b6707 WatchSource:0}: Error finding container d3a4334b2ef21e15529a99f7871fe92f0d39d3bb0346c08554cc50e7629b6707: Status 404 returned error can't find the container with id d3a4334b2ef21e15529a99f7871fe92f0d39d3bb0346c08554cc50e7629b6707
Sep 19 11:43:56.559680 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:56.559598    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m" event={"ID":"d152572a-a22d-4ccd-bb4e-8a8b02030e35","Type":"ContainerStarted","Data":"a0af193f41b54a77eaf33030630d287b13e7fe8a244c1d54107a7318ba5448df"}
Sep 19 11:43:56.559680 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:56.559645    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m" event={"ID":"d152572a-a22d-4ccd-bb4e-8a8b02030e35","Type":"ContainerStarted","Data":"d3a4334b2ef21e15529a99f7871fe92f0d39d3bb0346c08554cc50e7629b6707"}
Sep 19 11:43:56.559868 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:56.559737    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"
Sep 19 11:43:56.576332 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:56.576282    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m" podStartSLOduration=1.576264905 podStartE2EDuration="1.576264905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:43:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:43:56.575794406 +0000 UTC m=+588.498107288" watchObservedRunningTime="2026-09-19 11:43:56.576264905 +0000 UTC m=+588.498577785"
Sep 19 11:43:57.496429 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:57.496399    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"
Sep 19 11:43:58.506571 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:58.506539    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"
Sep 19 11:43:59.510578 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:43:59.510552    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"
Sep 19 11:44:00.520253 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:00.520224    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"
Sep 19 11:44:01.525107 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:01.525077    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"
Sep 19 11:44:01.525621 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:01.525154    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"
Sep 19 11:44:05.554987 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:05.554959    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"
Sep 19 11:44:05.637759 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:05.637727    2559 generic.go:356] "Generic (PLEG): container finished" podID="8109c4b3-2b3e-4f95-bf02-b26cdcda973d" containerID="2f2165ed2580b706bb3e95a59912847fc59788f718ee05061d7c9f9f4eda516e" exitCode=0
Sep 19 11:44:05.637937 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:05.637804    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72" event={"ID":"8109c4b3-2b3e-4f95-bf02-b26cdcda973d","Type":"ContainerDied","Data":"2f2165ed2580b706bb3e95a59912847fc59788f718ee05061d7c9f9f4eda516e"}
Sep 19 11:44:05.638277 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:05.638259    2559 scope.go:117] "RemoveContainer" containerID="2f2165ed2580b706bb3e95a59912847fc59788f718ee05061d7c9f9f4eda516e"
Sep 19 11:44:05.640387 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:05.640356    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:44:05.640487 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:05.640436    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:44:05.640552 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:05.640488    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:44:06.644517 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:06.644481    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72" event={"ID":"8109c4b3-2b3e-4f95-bf02-b26cdcda973d","Type":"ContainerStarted","Data":"dd21ad024f221bcccfc23377f83af8e3a9f77f0d512fcd3366975951f5a11612"}
Sep 19 11:44:07.523129 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:07.523091    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:44:07.523359 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:07.523288    2559 patch_prober.go:28] interesting pod/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 19 11:44:07.523359 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:07.523343    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72" podUID="8109c4b3-2b3e-4f95-bf02-b26cdcda973d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 19 11:44:07.568540 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:07.568516    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"
Sep 19 11:44:08.522698 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:08.522666    2559 patch_prober.go:28] interesting pod/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 19 11:44:08.523167 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:08.523138    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72" podUID="8109c4b3-2b3e-4f95-bf02-b26cdcda973d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 19 11:44:09.523026 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:09.522984    2559 patch_prober.go:28] interesting pod/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 19 11:44:09.523483 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:09.523057    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72" podUID="8109c4b3-2b3e-4f95-bf02-b26cdcda973d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 19 11:44:10.340077 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:10.340044    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb615d6f-srd47"]
Sep 19 11:44:10.345152 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:10.345131    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb615d6f-srd47"
Sep 19 11:44:10.347826 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:10.347795    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:44:10.351916 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:10.351891    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb615d6f-srd47"]
Sep 19 11:44:10.437985 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:10.437950    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd2f829a-f925-4268-b3c1-9c619e2f0fbd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cb615d6f-srd47\" (UID: \"dd2f829a-f925-4268-b3c1-9c619e2f0fbd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb615d6f-srd47"
Sep 19 11:44:10.523261 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:10.523228    2559 patch_prober.go:28] interesting pod/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 19 11:44:10.523722 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:10.523289    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72" podUID="8109c4b3-2b3e-4f95-bf02-b26cdcda973d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 19 11:44:10.538787 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:10.538760    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd2f829a-f925-4268-b3c1-9c619e2f0fbd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cb615d6f-srd47\" (UID: \"dd2f829a-f925-4268-b3c1-9c619e2f0fbd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb615d6f-srd47"
Sep 19 11:44:10.539417 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:10.539400    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd2f829a-f925-4268-b3c1-9c619e2f0fbd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cb615d6f-srd47\" (UID: \"dd2f829a-f925-4268-b3c1-9c619e2f0fbd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb615d6f-srd47"
Sep 19 11:44:10.659081 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:10.659001    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb615d6f-srd47"
Sep 19 11:44:10.962071 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:10.962051    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb615d6f-srd47"]
Sep 19 11:44:10.964014 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:44:10.963973    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd2f829a_f925_4268_b3c1_9c619e2f0fbd.slice/crio-ceb74c9902f1e9b5bfc06b0beb81f9390f42a1bfc226f415485f14c742aef330 WatchSource:0}: Error finding container ceb74c9902f1e9b5bfc06b0beb81f9390f42a1bfc226f415485f14c742aef330: Status 404 returned error can't find the container with id ceb74c9902f1e9b5bfc06b0beb81f9390f42a1bfc226f415485f14c742aef330
Sep 19 11:44:11.166415 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.166387    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"]
Sep 19 11:44:11.174547 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.174498    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"
Sep 19 11:44:11.177896 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.177338    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-d8d8bb\""
Sep 19 11:44:11.178582 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.178555    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"]
Sep 19 11:44:11.211259 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.206673    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"]
Sep 19 11:44:11.215929 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.211518    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"
Sep 19 11:44:11.221212 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.217055    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-d8d8bb-dockercfg-9x7g9\""
Sep 19 11:44:11.225609 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.225573    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"]
Sep 19 11:44:11.246228 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.246166    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c4l8\" (UniqueName: \"kubernetes.io/projected/a003bb0b-8221-4933-ab41-8fd3fc7ed5a5-kube-api-access-8c4l8\") pod \"payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg\" (UID: \"a003bb0b-8221-4933-ab41-8fd3fc7ed5a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"
Sep 19 11:44:11.246334 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.246261    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c54f9900-9aaf-4ac7-990d-d32b428e1a96-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c\" (UID: \"c54f9900-9aaf-4ac7-990d-d32b428e1a96\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"
Sep 19 11:44:11.246334 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.246300    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a003bb0b-8221-4933-ab41-8fd3fc7ed5a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg\" (UID: \"a003bb0b-8221-4933-ab41-8fd3fc7ed5a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"
Sep 19 11:44:11.246412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.246401    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxrqc\" (UniqueName: \"kubernetes.io/projected/c54f9900-9aaf-4ac7-990d-d32b428e1a96-kube-api-access-hxrqc\") pod \"payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c\" (UID: \"c54f9900-9aaf-4ac7-990d-d32b428e1a96\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"
Sep 19 11:44:11.347381 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.347338    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxrqc\" (UniqueName: \"kubernetes.io/projected/c54f9900-9aaf-4ac7-990d-d32b428e1a96-kube-api-access-hxrqc\") pod \"payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c\" (UID: \"c54f9900-9aaf-4ac7-990d-d32b428e1a96\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"
Sep 19 11:44:11.347552 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.347420    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c4l8\" (UniqueName: \"kubernetes.io/projected/a003bb0b-8221-4933-ab41-8fd3fc7ed5a5-kube-api-access-8c4l8\") pod \"payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg\" (UID: \"a003bb0b-8221-4933-ab41-8fd3fc7ed5a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"
Sep 19 11:44:11.347552 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.347465    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c54f9900-9aaf-4ac7-990d-d32b428e1a96-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c\" (UID: \"c54f9900-9aaf-4ac7-990d-d32b428e1a96\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"
Sep 19 11:44:11.347552 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.347491    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a003bb0b-8221-4933-ab41-8fd3fc7ed5a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg\" (UID: \"a003bb0b-8221-4933-ab41-8fd3fc7ed5a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"
Sep 19 11:44:11.348499 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.348472    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a003bb0b-8221-4933-ab41-8fd3fc7ed5a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg\" (UID: \"a003bb0b-8221-4933-ab41-8fd3fc7ed5a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"
Sep 19 11:44:11.348617 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.348510    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c54f9900-9aaf-4ac7-990d-d32b428e1a96-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c\" (UID: \"c54f9900-9aaf-4ac7-990d-d32b428e1a96\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"
Sep 19 11:44:11.355717 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.355693    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxrqc\" (UniqueName: \"kubernetes.io/projected/c54f9900-9aaf-4ac7-990d-d32b428e1a96-kube-api-access-hxrqc\") pod \"payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c\" (UID: \"c54f9900-9aaf-4ac7-990d-d32b428e1a96\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"
Sep 19 11:44:11.355826 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.355730    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c4l8\" (UniqueName: \"kubernetes.io/projected/a003bb0b-8221-4933-ab41-8fd3fc7ed5a5-kube-api-access-8c4l8\") pod \"payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg\" (UID: \"a003bb0b-8221-4933-ab41-8fd3fc7ed5a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"
Sep 19 11:44:11.506905 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.506805    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"
Sep 19 11:44:11.522890 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.522859    2559 patch_prober.go:28] interesting pod/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 19 11:44:11.523016 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.522926    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72" podUID="8109c4b3-2b3e-4f95-bf02-b26cdcda973d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 19 11:44:11.534474 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.534450    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"
Sep 19 11:44:11.678077 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.677998    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb615d6f-srd47" event={"ID":"dd2f829a-f925-4268-b3c1-9c619e2f0fbd","Type":"ContainerStarted","Data":"ceb74c9902f1e9b5bfc06b0beb81f9390f42a1bfc226f415485f14c742aef330"}
Sep 19 11:44:11.789787 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.789724    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"]
Sep 19 11:44:11.792779 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:44:11.792749    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda003bb0b_8221_4933_ab41_8fd3fc7ed5a5.slice/crio-1cc9aef6c258f101dae2d3ef7183ee993c9f31a9063e960b64e1c0173e3c227d WatchSource:0}: Error finding container 1cc9aef6c258f101dae2d3ef7183ee993c9f31a9063e960b64e1c0173e3c227d: Status 404 returned error can't find the container with id 1cc9aef6c258f101dae2d3ef7183ee993c9f31a9063e960b64e1c0173e3c227d
Sep 19 11:44:11.847912 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:11.847881    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"]
Sep 19 11:44:11.851711 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:44:11.851662    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc54f9900_9aaf_4ac7_990d_d32b428e1a96.slice/crio-77219c54e1b09c3577ca210127059b64d5d8c040953459cc04fe58eb7c17d920 WatchSource:0}: Error finding container 77219c54e1b09c3577ca210127059b64d5d8c040953459cc04fe58eb7c17d920: Status 404 returned error can't find the container with id 77219c54e1b09c3577ca210127059b64d5d8c040953459cc04fe58eb7c17d920
Sep 19 11:44:12.528830 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:12.528794    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:44:12.529173 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:12.529151    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:44:12.529994 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:12.529972    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:44:12.687558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:12.687515    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c" event={"ID":"c54f9900-9aaf-4ac7-990d-d32b428e1a96","Type":"ContainerStarted","Data":"9d6102d64c66893c3869807af4b55196556080188cbd3b337e5b54adc5088aa4"}
Sep 19 11:44:12.687998 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:12.687588    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c" event={"ID":"c54f9900-9aaf-4ac7-990d-d32b428e1a96","Type":"ContainerStarted","Data":"77219c54e1b09c3577ca210127059b64d5d8c040953459cc04fe58eb7c17d920"}
Sep 19 11:44:12.687998 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:12.687659    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"
Sep 19 11:44:12.689828 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:12.689790    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg" event={"ID":"a003bb0b-8221-4933-ab41-8fd3fc7ed5a5","Type":"ContainerStarted","Data":"11c7d078b1f0c5db872fc8229f46a59a35fa55fb566edea0f62a8e29c422b52b"}
Sep 19 11:44:12.689952 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:12.689833    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg" event={"ID":"a003bb0b-8221-4933-ab41-8fd3fc7ed5a5","Type":"ContainerStarted","Data":"1cc9aef6c258f101dae2d3ef7183ee993c9f31a9063e960b64e1c0173e3c227d"}
Sep 19 11:44:12.690014 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:12.689950    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"
Sep 19 11:44:12.709041 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:12.708983    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c" podStartSLOduration=1.7089654300000001 podStartE2EDuration="1.70896543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:44:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:44:12.705411444 +0000 UTC m=+604.627724340" watchObservedRunningTime="2026-09-19 11:44:12.70896543 +0000 UTC m=+604.631278309"
Sep 19 11:44:12.727915 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:12.726176    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg" podStartSLOduration=1.726156037 podStartE2EDuration="1.726156037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:44:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:44:12.725469802 +0000 UTC m=+604.647782665" watchObservedRunningTime="2026-09-19 11:44:12.726156037 +0000 UTC m=+604.648468913"
Sep 19 11:44:14.706601 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:14.706567    2559 generic.go:356] "Generic (PLEG): container finished" podID="dd2f829a-f925-4268-b3c1-9c619e2f0fbd" containerID="5df709d4c07abc29ecf3fedf966b831122935230ef8aa98a3c2a4863eebec540" exitCode=0
Sep 19 11:44:14.706970 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:14.706626    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb615d6f-srd47" event={"ID":"dd2f829a-f925-4268-b3c1-9c619e2f0fbd","Type":"ContainerDied","Data":"5df709d4c07abc29ecf3fedf966b831122935230ef8aa98a3c2a4863eebec540"}
Sep 19 11:44:15.858264 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:15.858242    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb615d6f-srd47"
Sep 19 11:44:15.897019 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:15.896992    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd2f829a-f925-4268-b3c1-9c619e2f0fbd-maas-api-service-ca\") pod \"dd2f829a-f925-4268-b3c1-9c619e2f0fbd\" (UID: \"dd2f829a-f925-4268-b3c1-9c619e2f0fbd\") "
Sep 19 11:44:15.897350 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:15.897326    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd2f829a-f925-4268-b3c1-9c619e2f0fbd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dd2f829a-f925-4268-b3c1-9c619e2f0fbd" (UID: "dd2f829a-f925-4268-b3c1-9c619e2f0fbd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:44:15.998021 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:15.997946    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd2f829a-f925-4268-b3c1-9c619e2f0fbd-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:44:16.718008 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:16.717980    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb615d6f-srd47"
Sep 19 11:44:16.718008 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:16.717995    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb615d6f-srd47" event={"ID":"dd2f829a-f925-4268-b3c1-9c619e2f0fbd","Type":"ContainerDied","Data":"ceb74c9902f1e9b5bfc06b0beb81f9390f42a1bfc226f415485f14c742aef330"}
Sep 19 11:44:16.718281 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:16.718040    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ceb74c9902f1e9b5bfc06b0beb81f9390f42a1bfc226f415485f14c742aef330"
Sep 19 11:44:20.846932 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:20.846896    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"]
Sep 19 11:44:20.847424 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:20.847131    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m" podUID="d152572a-a22d-4ccd-bb4e-8a8b02030e35" containerName="payload-processing" containerID="cri-o://a0af193f41b54a77eaf33030630d287b13e7fe8a244c1d54107a7318ba5448df" gracePeriod=30
Sep 19 11:44:21.018687 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.018655    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb615d6f-srd47"]
Sep 19 11:44:21.022501 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.022473    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb615d6f-srd47"]
Sep 19 11:44:21.115541 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.115474    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"
Sep 19 11:44:21.251090 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.251053    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24d6f\" (UniqueName: \"kubernetes.io/projected/d152572a-a22d-4ccd-bb4e-8a8b02030e35-kube-api-access-24d6f\") pod \"d152572a-a22d-4ccd-bb4e-8a8b02030e35\" (UID: \"d152572a-a22d-4ccd-bb4e-8a8b02030e35\") "
Sep 19 11:44:21.251310 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.251175    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d152572a-a22d-4ccd-bb4e-8a8b02030e35-plugins-config-volume\") pod \"d152572a-a22d-4ccd-bb4e-8a8b02030e35\" (UID: \"d152572a-a22d-4ccd-bb4e-8a8b02030e35\") "
Sep 19 11:44:21.251508 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.251485    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d152572a-a22d-4ccd-bb4e-8a8b02030e35-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d152572a-a22d-4ccd-bb4e-8a8b02030e35" (UID: "d152572a-a22d-4ccd-bb4e-8a8b02030e35"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:44:21.253441 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.253409    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d152572a-a22d-4ccd-bb4e-8a8b02030e35-kube-api-access-24d6f" (OuterVolumeSpecName: "kube-api-access-24d6f") pod "d152572a-a22d-4ccd-bb4e-8a8b02030e35" (UID: "d152572a-a22d-4ccd-bb4e-8a8b02030e35"). InnerVolumeSpecName "kube-api-access-24d6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:44:21.352866 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.352835    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d152572a-a22d-4ccd-bb4e-8a8b02030e35-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:44:21.352866 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.352861    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24d6f\" (UniqueName: \"kubernetes.io/projected/d152572a-a22d-4ccd-bb4e-8a8b02030e35-kube-api-access-24d6f\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:44:21.747333 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.747284    2559 generic.go:356] "Generic (PLEG): container finished" podID="d152572a-a22d-4ccd-bb4e-8a8b02030e35" containerID="a0af193f41b54a77eaf33030630d287b13e7fe8a244c1d54107a7318ba5448df" exitCode=0
Sep 19 11:44:21.747506 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.747361    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m" event={"ID":"d152572a-a22d-4ccd-bb4e-8a8b02030e35","Type":"ContainerDied","Data":"a0af193f41b54a77eaf33030630d287b13e7fe8a244c1d54107a7318ba5448df"}
Sep 19 11:44:21.747506 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.747422    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m" event={"ID":"d152572a-a22d-4ccd-bb4e-8a8b02030e35","Type":"ContainerDied","Data":"d3a4334b2ef21e15529a99f7871fe92f0d39d3bb0346c08554cc50e7629b6707"}
Sep 19 11:44:21.747506 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.747453    2559 scope.go:117] "RemoveContainer" containerID="a0af193f41b54a77eaf33030630d287b13e7fe8a244c1d54107a7318ba5448df"
Sep 19 11:44:21.747680 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.747611    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"
Sep 19 11:44:21.760011 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.759990    2559 scope.go:117] "RemoveContainer" containerID="a0af193f41b54a77eaf33030630d287b13e7fe8a244c1d54107a7318ba5448df"
Sep 19 11:44:21.760377 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:44:21.760356    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0af193f41b54a77eaf33030630d287b13e7fe8a244c1d54107a7318ba5448df\": container with ID starting with a0af193f41b54a77eaf33030630d287b13e7fe8a244c1d54107a7318ba5448df not found: ID does not exist" containerID="a0af193f41b54a77eaf33030630d287b13e7fe8a244c1d54107a7318ba5448df"
Sep 19 11:44:21.760443 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.760387    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0af193f41b54a77eaf33030630d287b13e7fe8a244c1d54107a7318ba5448df"} err="failed to get container status \"a0af193f41b54a77eaf33030630d287b13e7fe8a244c1d54107a7318ba5448df\": rpc error: code = NotFound desc = could not find container \"a0af193f41b54a77eaf33030630d287b13e7fe8a244c1d54107a7318ba5448df\": container with ID starting with a0af193f41b54a77eaf33030630d287b13e7fe8a244c1d54107a7318ba5448df not found: ID does not exist"
Sep 19 11:44:21.774491 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.774464    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"]
Sep 19 11:44:21.778287 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:21.778262    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cb615d6f-855999c645-rr62m"]
Sep 19 11:44:22.595369 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:22.595338    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d152572a-a22d-4ccd-bb4e-8a8b02030e35" path="/var/lib/kubelet/pods/d152572a-a22d-4ccd-bb4e-8a8b02030e35/volumes"
Sep 19 11:44:22.595728 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:22.595645    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd2f829a-f925-4268-b3c1-9c619e2f0fbd" path="/var/lib/kubelet/pods/dd2f829a-f925-4268-b3c1-9c619e2f0fbd/volumes"
Sep 19 11:44:23.700379 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:23.700351    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"
Sep 19 11:44:23.700815 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:23.700419    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"
Sep 19 11:44:28.240093 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.240058    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"]
Sep 19 11:44:28.240771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.240749    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d152572a-a22d-4ccd-bb4e-8a8b02030e35" containerName="payload-processing"
Sep 19 11:44:28.240906 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.240784    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d152572a-a22d-4ccd-bb4e-8a8b02030e35" containerName="payload-processing"
Sep 19 11:44:28.240906 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.240854    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd2f829a-f925-4268-b3c1-9c619e2f0fbd" containerName="revoke-keys"
Sep 19 11:44:28.240906 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.240864    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd2f829a-f925-4268-b3c1-9c619e2f0fbd" containerName="revoke-keys"
Sep 19 11:44:28.241065 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.241022    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d152572a-a22d-4ccd-bb4e-8a8b02030e35" containerName="payload-processing"
Sep 19 11:44:28.241065 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.241047    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd2f829a-f925-4268-b3c1-9c619e2f0fbd" containerName="revoke-keys"
Sep 19 11:44:28.245790 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.245756    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"
Sep 19 11:44:28.248333 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.248313    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-9cf8a188-dockercfg-fj6qc\""
Sep 19 11:44:28.248454 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.248374    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-9cf8a188\""
Sep 19 11:44:28.252724 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.252700    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"]
Sep 19 11:44:28.323559 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.323521    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2bgg\" (UniqueName: \"kubernetes.io/projected/de333eb7-e948-4815-9cf5-6c35cf9d4332-kube-api-access-j2bgg\") pod \"payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs\" (UID: \"de333eb7-e948-4815-9cf5-6c35cf9d4332\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"
Sep 19 11:44:28.323716 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.323666    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de333eb7-e948-4815-9cf5-6c35cf9d4332-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs\" (UID: \"de333eb7-e948-4815-9cf5-6c35cf9d4332\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"
Sep 19 11:44:28.424937 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.424905    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2bgg\" (UniqueName: \"kubernetes.io/projected/de333eb7-e948-4815-9cf5-6c35cf9d4332-kube-api-access-j2bgg\") pod \"payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs\" (UID: \"de333eb7-e948-4815-9cf5-6c35cf9d4332\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"
Sep 19 11:44:28.425101 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.424986    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de333eb7-e948-4815-9cf5-6c35cf9d4332-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs\" (UID: \"de333eb7-e948-4815-9cf5-6c35cf9d4332\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"
Sep 19 11:44:28.425700 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.425682    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de333eb7-e948-4815-9cf5-6c35cf9d4332-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs\" (UID: \"de333eb7-e948-4815-9cf5-6c35cf9d4332\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"
Sep 19 11:44:28.432816 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.432790    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2bgg\" (UniqueName: \"kubernetes.io/projected/de333eb7-e948-4815-9cf5-6c35cf9d4332-kube-api-access-j2bgg\") pod \"payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs\" (UID: \"de333eb7-e948-4815-9cf5-6c35cf9d4332\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"
Sep 19 11:44:28.558545 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.558470    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"
Sep 19 11:44:28.831882 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:28.831858    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"]
Sep 19 11:44:28.833839 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:44:28.833815    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde333eb7_e948_4815_9cf5_6c35cf9d4332.slice/crio-e6ea7004dd549043d6824a3b0d7efef3e44ceb3e92fc583e9af5cb1cef7900e9 WatchSource:0}: Error finding container e6ea7004dd549043d6824a3b0d7efef3e44ceb3e92fc583e9af5cb1cef7900e9: Status 404 returned error can't find the container with id e6ea7004dd549043d6824a3b0d7efef3e44ceb3e92fc583e9af5cb1cef7900e9
Sep 19 11:44:29.794735 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:29.794660    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs" event={"ID":"de333eb7-e948-4815-9cf5-6c35cf9d4332","Type":"ContainerStarted","Data":"5d14177dd42e52994e7b31d9b24fd2ec6cd65b430394dcc2cfefffc233889b2f"}
Sep 19 11:44:29.794735 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:29.794707    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs" event={"ID":"de333eb7-e948-4815-9cf5-6c35cf9d4332","Type":"ContainerStarted","Data":"e6ea7004dd549043d6824a3b0d7efef3e44ceb3e92fc583e9af5cb1cef7900e9"}
Sep 19 11:44:29.795130 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:29.794876    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"
Sep 19 11:44:40.803563 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:40.803522    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"
Sep 19 11:44:40.819466 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:40.819421    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs" podStartSLOduration=12.819406842 podStartE2EDuration="12.819406842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:44:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:44:29.80842466 +0000 UTC m=+621.730737542" watchObservedRunningTime="2026-09-19 11:44:40.819406842 +0000 UTC m=+632.741719752"
Sep 19 11:44:54.481968 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:54.481927    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8"]
Sep 19 11:44:54.486282 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:54.486256    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8"
Sep 19 11:44:54.489625 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:54.489601    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:44:54.492636 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:54.492598    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8"]
Sep 19 11:44:54.572647 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:54.572610    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66aa081d-c4dd-4a99-b0c4-8510e11dd77f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8\" (UID: \"66aa081d-c4dd-4a99-b0c4-8510e11dd77f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8"
Sep 19 11:44:54.674105 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:54.674073    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66aa081d-c4dd-4a99-b0c4-8510e11dd77f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8\" (UID: \"66aa081d-c4dd-4a99-b0c4-8510e11dd77f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8"
Sep 19 11:44:54.674746 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:54.674723    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66aa081d-c4dd-4a99-b0c4-8510e11dd77f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8\" (UID: \"66aa081d-c4dd-4a99-b0c4-8510e11dd77f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8"
Sep 19 11:44:54.805578 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:54.805485    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8"
Sep 19 11:44:55.073827 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:55.073803    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8"]
Sep 19 11:44:55.075868 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:44:55.075838    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66aa081d_c4dd_4a99_b0c4_8510e11dd77f.slice/crio-3ca98caf3c9218447054ef721188f8a9025de9d9522f29ea259f1a1ab63713d9 WatchSource:0}: Error finding container 3ca98caf3c9218447054ef721188f8a9025de9d9522f29ea259f1a1ab63713d9: Status 404 returned error can't find the container with id 3ca98caf3c9218447054ef721188f8a9025de9d9522f29ea259f1a1ab63713d9
Sep 19 11:44:55.951680 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:55.951649    2559 generic.go:356] "Generic (PLEG): container finished" podID="66aa081d-c4dd-4a99-b0c4-8510e11dd77f" containerID="373b035c74b5aa2d5cd2f11ca19137b0f8a2bed1de7f5ac46d15e70ebdf3b96c" exitCode=0
Sep 19 11:44:55.952103 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:55.951692    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8" event={"ID":"66aa081d-c4dd-4a99-b0c4-8510e11dd77f","Type":"ContainerDied","Data":"373b035c74b5aa2d5cd2f11ca19137b0f8a2bed1de7f5ac46d15e70ebdf3b96c"}
Sep 19 11:44:55.952103 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:55.951747    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8" event={"ID":"66aa081d-c4dd-4a99-b0c4-8510e11dd77f","Type":"ContainerStarted","Data":"3ca98caf3c9218447054ef721188f8a9025de9d9522f29ea259f1a1ab63713d9"}
Sep 19 11:44:57.097888 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:57.097863    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8"
Sep 19 11:44:57.198313 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:57.198283    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66aa081d-c4dd-4a99-b0c4-8510e11dd77f-maas-api-service-ca\") pod \"66aa081d-c4dd-4a99-b0c4-8510e11dd77f\" (UID: \"66aa081d-c4dd-4a99-b0c4-8510e11dd77f\") "
Sep 19 11:44:57.198599 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:57.198578    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66aa081d-c4dd-4a99-b0c4-8510e11dd77f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66aa081d-c4dd-4a99-b0c4-8510e11dd77f" (UID: "66aa081d-c4dd-4a99-b0c4-8510e11dd77f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:44:57.299142 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:57.299067    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66aa081d-c4dd-4a99-b0c4-8510e11dd77f-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:44:57.963729 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:57.963694    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8"
Sep 19 11:44:57.963892 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:57.963740    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8" event={"ID":"66aa081d-c4dd-4a99-b0c4-8510e11dd77f","Type":"ContainerDied","Data":"3ca98caf3c9218447054ef721188f8a9025de9d9522f29ea259f1a1ab63713d9"}
Sep 19 11:44:57.963892 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:44:57.963790    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ca98caf3c9218447054ef721188f8a9025de9d9522f29ea259f1a1ab63713d9"
Sep 19 11:45:00.137454 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.137421    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d8d8bb-29830305-778hr"]
Sep 19 11:45:00.137884 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.137870    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66aa081d-c4dd-4a99-b0c4-8510e11dd77f" containerName="revoke-keys"
Sep 19 11:45:00.137884 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.137885    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="66aa081d-c4dd-4a99-b0c4-8510e11dd77f" containerName="revoke-keys"
Sep 19 11:45:00.137997 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.137988    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="66aa081d-c4dd-4a99-b0c4-8510e11dd77f" containerName="revoke-keys"
Sep 19 11:45:00.141673 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.141649    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7eb15e-29830305-mfn45"]
Sep 19 11:45:00.141826 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.141771    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d8d8bb-29830305-778hr"
Sep 19 11:45:00.144345 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.144323    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:45:00.145520 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.145503    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9cf8a188-29830305-6685l"]
Sep 19 11:45:00.145619 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.145602    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7eb15e-29830305-mfn45"
Sep 19 11:45:00.149484 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.149461    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9cf8a188-29830305-6685l"
Sep 19 11:45:00.155980 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.155955    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d8d8bb-29830305-778hr"
Sep 19 11:45:00.156163 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.156146    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d8d8bb-29830305-778hr"]
Sep 19 11:45:00.158819 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.158729    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7eb15e-29830305-mfn45"]
Sep 19 11:45:00.171789 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.171753    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29830305-g7662"]
Sep 19 11:45:00.179315 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.179289    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7eb15e-29830305-mfn45"
Sep 19 11:45:00.191409 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.191327    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29830305-g7662"]
Sep 19 11:45:00.191531 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.191444    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9cf8a188-29830305-6685l"]
Sep 19 11:45:00.191605 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.191595    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830305-g7662"
Sep 19 11:45:00.224525 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.224499    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9cf8a188-29830305-6685l"
Sep 19 11:45:00.238701 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.237966    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830305-g7662"
Sep 19 11:45:00.459262 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.459241    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d8d8bb-29830305-778hr"]
Sep 19 11:45:00.461636 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:45:00.461608    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c27a66b_d680_4f5d_8d28_6b632e69c131.slice/crio-4609a37a291277915bfb1bdb884e48be1bcf210fc70929a8fd051313e1841a5e WatchSource:0}: Error finding container 4609a37a291277915bfb1bdb884e48be1bcf210fc70929a8fd051313e1841a5e: Status 404 returned error can't find the container with id 4609a37a291277915bfb1bdb884e48be1bcf210fc70929a8fd051313e1841a5e
Sep 19 11:45:00.473361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.471436    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7eb15e-29830305-mfn45"]
Sep 19 11:45:00.477963 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:45:00.477937    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945af994_c1f0_4aaa_a01b_75e84b8d0b77.slice/crio-1bf23cbed5fc5ed10bade411f4efc89804f759e043705bcf2e94b34e7af007e8 WatchSource:0}: Error finding container 1bf23cbed5fc5ed10bade411f4efc89804f759e043705bcf2e94b34e7af007e8: Status 404 returned error can't find the container with id 1bf23cbed5fc5ed10bade411f4efc89804f759e043705bcf2e94b34e7af007e8
Sep 19 11:45:00.529850 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.529821    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29830305-g7662"]
Sep 19 11:45:00.531413 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:45:00.531343    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6af369db_935f_4496_8917_05fcd1a0deac.slice/crio-64644d38fd6659d57b7a5d9ebe853acee49995232389c712e41358dbd6931e77 WatchSource:0}: Error finding container 64644d38fd6659d57b7a5d9ebe853acee49995232389c712e41358dbd6931e77: Status 404 returned error can't find the container with id 64644d38fd6659d57b7a5d9ebe853acee49995232389c712e41358dbd6931e77
Sep 19 11:45:00.533521 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.533498    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9cf8a188-29830305-6685l"]
Sep 19 11:45:00.533588 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:45:00.533516    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e14fcfd_2bbc_49c1_9814_82214dc05304.slice/crio-7c70bbe32c56b72eea560909a46904882bbb76df031f9275cdf952d0b4958c22 WatchSource:0}: Error finding container 7c70bbe32c56b72eea560909a46904882bbb76df031f9275cdf952d0b4958c22: Status 404 returned error can't find the container with id 7c70bbe32c56b72eea560909a46904882bbb76df031f9275cdf952d0b4958c22
Sep 19 11:45:00.982935 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.982867    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9cf8a188-29830305-6685l" event={"ID":"9e14fcfd-2bbc-49c1-9814-82214dc05304","Type":"ContainerStarted","Data":"7c70bbe32c56b72eea560909a46904882bbb76df031f9275cdf952d0b4958c22"}
Sep 19 11:45:00.985049 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.985019    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7eb15e-29830305-mfn45" event={"ID":"945af994-c1f0-4aaa-a01b-75e84b8d0b77","Type":"ContainerStarted","Data":"ceaa412893cff7be7cffa324fe17af3b5c60ccd091e7060055cbbcfe36ed7832"}
Sep 19 11:45:00.985178 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.985088    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7eb15e-29830305-mfn45" event={"ID":"945af994-c1f0-4aaa-a01b-75e84b8d0b77","Type":"ContainerStarted","Data":"1bf23cbed5fc5ed10bade411f4efc89804f759e043705bcf2e94b34e7af007e8"}
Sep 19 11:45:00.986960 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.986936    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d8d8bb-29830305-778hr" event={"ID":"4c27a66b-d680-4f5d-8d28-6b632e69c131","Type":"ContainerStarted","Data":"e239220aecf16d48f18a0c8370df89d5b77fd8000a84c71e7386adef09305e64"}
Sep 19 11:45:00.987065 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.986984    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d8d8bb-29830305-778hr" event={"ID":"4c27a66b-d680-4f5d-8d28-6b632e69c131","Type":"ContainerStarted","Data":"4609a37a291277915bfb1bdb884e48be1bcf210fc70929a8fd051313e1841a5e"}
Sep 19 11:45:00.988886 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.988863    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830305-g7662" event={"ID":"6af369db-935f-4496-8917-05fcd1a0deac","Type":"ContainerStarted","Data":"81fb308517babca759d47c813c23a7dd350aaf6c29ca85f5dfecf8775912c612"}
Sep 19 11:45:00.989098 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:00.988912    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830305-g7662" event={"ID":"6af369db-935f-4496-8917-05fcd1a0deac","Type":"ContainerStarted","Data":"64644d38fd6659d57b7a5d9ebe853acee49995232389c712e41358dbd6931e77"}
Sep 19 11:45:01.000397 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:01.000363    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7eb15e-29830305-mfn45" podStartSLOduration=1.000349302 podStartE2EDuration="1.000349302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:45:00.99891939 +0000 UTC m=+652.921232283" watchObservedRunningTime="2026-09-19 11:45:01.000349302 +0000 UTC m=+652.922662177"
Sep 19 11:45:01.013234 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:01.013159    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d8d8bb-29830305-778hr" podStartSLOduration=1.013146193 podStartE2EDuration="1.013146193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:45:01.011789801 +0000 UTC m=+652.934102683" watchObservedRunningTime="2026-09-19 11:45:01.013146193 +0000 UTC m=+652.935459074"
Sep 19 11:45:01.026080 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:01.026039    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830305-g7662" podStartSLOduration=1.026026822 podStartE2EDuration="1.026026822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:45:01.024048547 +0000 UTC m=+652.946361431" watchObservedRunningTime="2026-09-19 11:45:01.026026822 +0000 UTC m=+652.948339703"
Sep 19 11:45:01.998876 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:01.998787    2559 generic.go:356] "Generic (PLEG): container finished" podID="945af994-c1f0-4aaa-a01b-75e84b8d0b77" containerID="ceaa412893cff7be7cffa324fe17af3b5c60ccd091e7060055cbbcfe36ed7832" exitCode=0
Sep 19 11:45:01.999265 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:01.998882    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7eb15e-29830305-mfn45" event={"ID":"945af994-c1f0-4aaa-a01b-75e84b8d0b77","Type":"ContainerDied","Data":"ceaa412893cff7be7cffa324fe17af3b5c60ccd091e7060055cbbcfe36ed7832"}
Sep 19 11:45:02.000535 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:02.000508    2559 generic.go:356] "Generic (PLEG): container finished" podID="4c27a66b-d680-4f5d-8d28-6b632e69c131" containerID="e239220aecf16d48f18a0c8370df89d5b77fd8000a84c71e7386adef09305e64" exitCode=0
Sep 19 11:45:02.000659 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:02.000592    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d8d8bb-29830305-778hr" event={"ID":"4c27a66b-d680-4f5d-8d28-6b632e69c131","Type":"ContainerDied","Data":"e239220aecf16d48f18a0c8370df89d5b77fd8000a84c71e7386adef09305e64"}
Sep 19 11:45:02.002291 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:02.002270    2559 generic.go:356] "Generic (PLEG): container finished" podID="6af369db-935f-4496-8917-05fcd1a0deac" containerID="81fb308517babca759d47c813c23a7dd350aaf6c29ca85f5dfecf8775912c612" exitCode=0
Sep 19 11:45:02.002416 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:02.002350    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830305-g7662" event={"ID":"6af369db-935f-4496-8917-05fcd1a0deac","Type":"ContainerDied","Data":"81fb308517babca759d47c813c23a7dd350aaf6c29ca85f5dfecf8775912c612"}
Sep 19 11:45:02.003839 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:02.003817    2559 generic.go:356] "Generic (PLEG): container finished" podID="9e14fcfd-2bbc-49c1-9814-82214dc05304" containerID="2b7e264168658edd264fd38a8fbb5bac9a1086a1708bd8af207d5f403d6d1374" exitCode=0
Sep 19 11:45:02.003929 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:02.003858    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9cf8a188-29830305-6685l" event={"ID":"9e14fcfd-2bbc-49c1-9814-82214dc05304","Type":"ContainerDied","Data":"2b7e264168658edd264fd38a8fbb5bac9a1086a1708bd8af207d5f403d6d1374"}
Sep 19 11:45:03.176320 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:03.176292    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7eb15e-29830305-mfn45"
Sep 19 11:45:03.180918 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:03.180895    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9cf8a188-29830305-6685l"
Sep 19 11:45:03.237515 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:03.237495    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830305-g7662"
Sep 19 11:45:03.240537 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:03.240519    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d8d8bb-29830305-778hr"
Sep 19 11:45:04.017792 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:04.017768    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9cf8a188-29830305-6685l"
Sep 19 11:45:04.017792 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:04.017784    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9cf8a188-29830305-6685l" event={"ID":"9e14fcfd-2bbc-49c1-9814-82214dc05304","Type":"ContainerDied","Data":"7c70bbe32c56b72eea560909a46904882bbb76df031f9275cdf952d0b4958c22"}
Sep 19 11:45:04.018008 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:04.017825    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c70bbe32c56b72eea560909a46904882bbb76df031f9275cdf952d0b4958c22"
Sep 19 11:45:04.019310 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:04.019282    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7eb15e-29830305-mfn45"
Sep 19 11:45:04.019459 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:04.019280    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7eb15e-29830305-mfn45" event={"ID":"945af994-c1f0-4aaa-a01b-75e84b8d0b77","Type":"ContainerDied","Data":"1bf23cbed5fc5ed10bade411f4efc89804f759e043705bcf2e94b34e7af007e8"}
Sep 19 11:45:04.019459 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:04.019398    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1bf23cbed5fc5ed10bade411f4efc89804f759e043705bcf2e94b34e7af007e8"
Sep 19 11:45:04.020580 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:04.020551    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d8d8bb-29830305-778hr"
Sep 19 11:45:04.020717 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:04.020588    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d8d8bb-29830305-778hr" event={"ID":"4c27a66b-d680-4f5d-8d28-6b632e69c131","Type":"ContainerDied","Data":"4609a37a291277915bfb1bdb884e48be1bcf210fc70929a8fd051313e1841a5e"}
Sep 19 11:45:04.020717 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:04.020629    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4609a37a291277915bfb1bdb884e48be1bcf210fc70929a8fd051313e1841a5e"
Sep 19 11:45:04.022092 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:04.022068    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830305-g7662" event={"ID":"6af369db-935f-4496-8917-05fcd1a0deac","Type":"ContainerDied","Data":"64644d38fd6659d57b7a5d9ebe853acee49995232389c712e41358dbd6931e77"}
Sep 19 11:45:04.022244 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:04.022110    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64644d38fd6659d57b7a5d9ebe853acee49995232389c712e41358dbd6931e77"
Sep 19 11:45:04.022244 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:04.022075    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830305-g7662"
Sep 19 11:45:05.940099 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:05.940066    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9cf8a188-29830305-6685l"]
Sep 19 11:45:05.942380 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:05.942348    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9cf8a188-29830305-6685l"]
Sep 19 11:45:05.967757 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:05.967727    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"]
Sep 19 11:45:05.967990 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:05.967961    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs" podUID="de333eb7-e948-4815-9cf5-6c35cf9d4332" containerName="payload-pre-processing" containerID="cri-o://5d14177dd42e52994e7b31d9b24fd2ec6cd65b430394dcc2cfefffc233889b2f" gracePeriod=30
Sep 19 11:45:06.132752 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:06.132715    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8"]
Sep 19 11:45:06.135813 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:06.135782    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9cf8a188-sfdx8"]
Sep 19 11:45:06.229008 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:06.228986    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"
Sep 19 11:45:06.383870 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:06.383837    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2bgg\" (UniqueName: \"kubernetes.io/projected/de333eb7-e948-4815-9cf5-6c35cf9d4332-kube-api-access-j2bgg\") pod \"de333eb7-e948-4815-9cf5-6c35cf9d4332\" (UID: \"de333eb7-e948-4815-9cf5-6c35cf9d4332\") "
Sep 19 11:45:06.384037 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:06.383906    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de333eb7-e948-4815-9cf5-6c35cf9d4332-plugins-config-volume\") pod \"de333eb7-e948-4815-9cf5-6c35cf9d4332\" (UID: \"de333eb7-e948-4815-9cf5-6c35cf9d4332\") "
Sep 19 11:45:06.384319 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:06.384290    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de333eb7-e948-4815-9cf5-6c35cf9d4332-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "de333eb7-e948-4815-9cf5-6c35cf9d4332" (UID: "de333eb7-e948-4815-9cf5-6c35cf9d4332"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:45:06.386115 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:06.386084    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de333eb7-e948-4815-9cf5-6c35cf9d4332-kube-api-access-j2bgg" (OuterVolumeSpecName: "kube-api-access-j2bgg") pod "de333eb7-e948-4815-9cf5-6c35cf9d4332" (UID: "de333eb7-e948-4815-9cf5-6c35cf9d4332"). InnerVolumeSpecName "kube-api-access-j2bgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:45:06.484690 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:06.484602    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2bgg\" (UniqueName: \"kubernetes.io/projected/de333eb7-e948-4815-9cf5-6c35cf9d4332-kube-api-access-j2bgg\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:45:06.484690 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:06.484632    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de333eb7-e948-4815-9cf5-6c35cf9d4332-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:45:06.596044 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:06.596009    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66aa081d-c4dd-4a99-b0c4-8510e11dd77f" path="/var/lib/kubelet/pods/66aa081d-c4dd-4a99-b0c4-8510e11dd77f/volumes"
Sep 19 11:45:06.596379 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:06.596363    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e14fcfd-2bbc-49c1-9814-82214dc05304" path="/var/lib/kubelet/pods/9e14fcfd-2bbc-49c1-9814-82214dc05304/volumes"
Sep 19 11:45:07.042073 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:07.042038    2559 generic.go:356] "Generic (PLEG): container finished" podID="de333eb7-e948-4815-9cf5-6c35cf9d4332" containerID="5d14177dd42e52994e7b31d9b24fd2ec6cd65b430394dcc2cfefffc233889b2f" exitCode=0
Sep 19 11:45:07.042502 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:07.042083    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"
Sep 19 11:45:07.042502 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:07.042112    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs" event={"ID":"de333eb7-e948-4815-9cf5-6c35cf9d4332","Type":"ContainerDied","Data":"5d14177dd42e52994e7b31d9b24fd2ec6cd65b430394dcc2cfefffc233889b2f"}
Sep 19 11:45:07.042502 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:07.042170    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs" event={"ID":"de333eb7-e948-4815-9cf5-6c35cf9d4332","Type":"ContainerDied","Data":"e6ea7004dd549043d6824a3b0d7efef3e44ceb3e92fc583e9af5cb1cef7900e9"}
Sep 19 11:45:07.042502 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:07.042226    2559 scope.go:117] "RemoveContainer" containerID="5d14177dd42e52994e7b31d9b24fd2ec6cd65b430394dcc2cfefffc233889b2f"
Sep 19 11:45:07.052954 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:07.052936    2559 scope.go:117] "RemoveContainer" containerID="5d14177dd42e52994e7b31d9b24fd2ec6cd65b430394dcc2cfefffc233889b2f"
Sep 19 11:45:07.053321 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:45:07.053290    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d14177dd42e52994e7b31d9b24fd2ec6cd65b430394dcc2cfefffc233889b2f\": container with ID starting with 5d14177dd42e52994e7b31d9b24fd2ec6cd65b430394dcc2cfefffc233889b2f not found: ID does not exist" containerID="5d14177dd42e52994e7b31d9b24fd2ec6cd65b430394dcc2cfefffc233889b2f"
Sep 19 11:45:07.053409 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:07.053320    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d14177dd42e52994e7b31d9b24fd2ec6cd65b430394dcc2cfefffc233889b2f"} err="failed to get container status \"5d14177dd42e52994e7b31d9b24fd2ec6cd65b430394dcc2cfefffc233889b2f\": rpc error: code = NotFound desc = could not find container \"5d14177dd42e52994e7b31d9b24fd2ec6cd65b430394dcc2cfefffc233889b2f\": container with ID starting with 5d14177dd42e52994e7b31d9b24fd2ec6cd65b430394dcc2cfefffc233889b2f not found: ID does not exist"
Sep 19 11:45:07.062496 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:07.062474    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"]
Sep 19 11:45:07.066459 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:07.066438    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9cf8a188-7f6bd99fc8-njbxs"]
Sep 19 11:45:08.595772 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:08.595745    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de333eb7-e948-4815-9cf5-6c35cf9d4332" path="/var/lib/kubelet/pods/de333eb7-e948-4815-9cf5-6c35cf9d4332/volumes"
Sep 19 11:45:11.773600 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.773567    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"]
Sep 19 11:45:11.774055 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.774042    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de333eb7-e948-4815-9cf5-6c35cf9d4332" containerName="payload-pre-processing"
Sep 19 11:45:11.774110 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.774057    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="de333eb7-e948-4815-9cf5-6c35cf9d4332" containerName="payload-pre-processing"
Sep 19 11:45:11.774110 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.774070    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6af369db-935f-4496-8917-05fcd1a0deac" containerName="cleanup"
Sep 19 11:45:11.774110 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.774076    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af369db-935f-4496-8917-05fcd1a0deac" containerName="cleanup"
Sep 19 11:45:11.774110 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.774090    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c27a66b-d680-4f5d-8d28-6b632e69c131" containerName="cleanup"
Sep 19 11:45:11.774110 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.774098    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c27a66b-d680-4f5d-8d28-6b632e69c131" containerName="cleanup"
Sep 19 11:45:11.774110 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.774107    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e14fcfd-2bbc-49c1-9814-82214dc05304" containerName="cleanup"
Sep 19 11:45:11.774110 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.774113    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e14fcfd-2bbc-49c1-9814-82214dc05304" containerName="cleanup"
Sep 19 11:45:11.774483 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.774140    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="945af994-c1f0-4aaa-a01b-75e84b8d0b77" containerName="cleanup"
Sep 19 11:45:11.774483 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.774148    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="945af994-c1f0-4aaa-a01b-75e84b8d0b77" containerName="cleanup"
Sep 19 11:45:11.774483 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.774293    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="de333eb7-e948-4815-9cf5-6c35cf9d4332" containerName="payload-pre-processing"
Sep 19 11:45:11.774483 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.774309    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c27a66b-d680-4f5d-8d28-6b632e69c131" containerName="cleanup"
Sep 19 11:45:11.774483 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.774322    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6af369db-935f-4496-8917-05fcd1a0deac" containerName="cleanup"
Sep 19 11:45:11.774483 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.774333    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="945af994-c1f0-4aaa-a01b-75e84b8d0b77" containerName="cleanup"
Sep 19 11:45:11.774483 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.774344    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e14fcfd-2bbc-49c1-9814-82214dc05304" containerName="cleanup"
Sep 19 11:45:11.778553 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.778534    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"
Sep 19 11:45:11.781468 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.781441    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5deb747f-dockercfg-qtjlc\""
Sep 19 11:45:11.781603 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.781445    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5deb747f\""
Sep 19 11:45:11.789384 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.789364    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"]
Sep 19 11:45:11.938442 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.938414    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4984308b-4cbc-46d2-867e-df5e77e43a0a-plugins-config-volume\") pod \"payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q\" (UID: \"4984308b-4cbc-46d2-867e-df5e77e43a0a\") " pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"
Sep 19 11:45:11.938602 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:11.938529    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c72q\" (UniqueName: \"kubernetes.io/projected/4984308b-4cbc-46d2-867e-df5e77e43a0a-kube-api-access-9c72q\") pod \"payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q\" (UID: \"4984308b-4cbc-46d2-867e-df5e77e43a0a\") " pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"
Sep 19 11:45:12.039316 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:12.039236    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c72q\" (UniqueName: \"kubernetes.io/projected/4984308b-4cbc-46d2-867e-df5e77e43a0a-kube-api-access-9c72q\") pod \"payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q\" (UID: \"4984308b-4cbc-46d2-867e-df5e77e43a0a\") " pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"
Sep 19 11:45:12.039316 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:12.039302    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4984308b-4cbc-46d2-867e-df5e77e43a0a-plugins-config-volume\") pod \"payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q\" (UID: \"4984308b-4cbc-46d2-867e-df5e77e43a0a\") " pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"
Sep 19 11:45:12.039967 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:12.039947    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4984308b-4cbc-46d2-867e-df5e77e43a0a-plugins-config-volume\") pod \"payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q\" (UID: \"4984308b-4cbc-46d2-867e-df5e77e43a0a\") " pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"
Sep 19 11:45:12.046455 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:12.046435    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c72q\" (UniqueName: \"kubernetes.io/projected/4984308b-4cbc-46d2-867e-df5e77e43a0a-kube-api-access-9c72q\") pod \"payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q\" (UID: \"4984308b-4cbc-46d2-867e-df5e77e43a0a\") " pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"
Sep 19 11:45:12.090829 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:12.090801    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"
Sep 19 11:45:12.365966 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:12.365937    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"]
Sep 19 11:45:12.366960 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:45:12.366932    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4984308b_4cbc_46d2_867e_df5e77e43a0a.slice/crio-7464a09888c09c1749445539b4e198cafe95a69eb811ee529301234bbbcb5771 WatchSource:0}: Error finding container 7464a09888c09c1749445539b4e198cafe95a69eb811ee529301234bbbcb5771: Status 404 returned error can't find the container with id 7464a09888c09c1749445539b4e198cafe95a69eb811ee529301234bbbcb5771
Sep 19 11:45:13.080264 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:13.080228    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q" event={"ID":"4984308b-4cbc-46d2-867e-df5e77e43a0a","Type":"ContainerStarted","Data":"0ab00ea4319cf822e4ee8a62182552a73a35f0c1ad19917f1551b63dd141db7a"}
Sep 19 11:45:13.080670 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:13.080294    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q" event={"ID":"4984308b-4cbc-46d2-867e-df5e77e43a0a","Type":"ContainerStarted","Data":"7464a09888c09c1749445539b4e198cafe95a69eb811ee529301234bbbcb5771"}
Sep 19 11:45:13.080670 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:13.080448    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"
Sep 19 11:45:13.098703 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:13.098649    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q" podStartSLOduration=2.098632391 podStartE2EDuration="2.098632391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:45:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:45:13.097146991 +0000 UTC m=+665.019459874" watchObservedRunningTime="2026-09-19 11:45:13.098632391 +0000 UTC m=+665.020945275"
Sep 19 11:45:24.088043 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:24.088013    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"
Sep 19 11:45:35.615156 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:35.615126    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5deb747f-8hp75"]
Sep 19 11:45:35.618991 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:35.618965    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5deb747f-8hp75"
Sep 19 11:45:35.621333 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:35.621314    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:45:35.627345 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:35.627321    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5deb747f-8hp75"]
Sep 19 11:45:35.669528 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:35.669502    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc27ee59-8e16-4bf8-84ed-3ff4ebab192b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5deb747f-8hp75\" (UID: \"bc27ee59-8e16-4bf8-84ed-3ff4ebab192b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5deb747f-8hp75"
Sep 19 11:45:35.770392 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:35.770357    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc27ee59-8e16-4bf8-84ed-3ff4ebab192b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5deb747f-8hp75\" (UID: \"bc27ee59-8e16-4bf8-84ed-3ff4ebab192b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5deb747f-8hp75"
Sep 19 11:45:35.771003 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:35.770969    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc27ee59-8e16-4bf8-84ed-3ff4ebab192b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5deb747f-8hp75\" (UID: \"bc27ee59-8e16-4bf8-84ed-3ff4ebab192b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5deb747f-8hp75"
Sep 19 11:45:35.930580 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:35.930555    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5deb747f-8hp75"
Sep 19 11:45:36.200473 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:36.200450    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5deb747f-8hp75"]
Sep 19 11:45:36.202609 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:45:36.202567    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc27ee59_8e16_4bf8_84ed_3ff4ebab192b.slice/crio-f022eb669dc9c0c31f2d6b98d39c21dbaf1d6a9ad035a30d82fda1281645ebfd WatchSource:0}: Error finding container f022eb669dc9c0c31f2d6b98d39c21dbaf1d6a9ad035a30d82fda1281645ebfd: Status 404 returned error can't find the container with id f022eb669dc9c0c31f2d6b98d39c21dbaf1d6a9ad035a30d82fda1281645ebfd
Sep 19 11:45:36.216700 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:36.216676    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5deb747f-8hp75" event={"ID":"bc27ee59-8e16-4bf8-84ed-3ff4ebab192b","Type":"ContainerStarted","Data":"f022eb669dc9c0c31f2d6b98d39c21dbaf1d6a9ad035a30d82fda1281645ebfd"}
Sep 19 11:45:37.224132 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:37.224098    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc27ee59-8e16-4bf8-84ed-3ff4ebab192b" containerID="12a98a386dbbf47746c93172d19d2b27dca3b90215d8e37cbb763deb0c9288d3" exitCode=0
Sep 19 11:45:37.224532 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:37.224208    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5deb747f-8hp75" event={"ID":"bc27ee59-8e16-4bf8-84ed-3ff4ebab192b","Type":"ContainerDied","Data":"12a98a386dbbf47746c93172d19d2b27dca3b90215d8e37cbb763deb0c9288d3"}
Sep 19 11:45:38.370534 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:38.370503    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5deb747f-8hp75"
Sep 19 11:45:38.496103 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:38.496002    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc27ee59-8e16-4bf8-84ed-3ff4ebab192b-maas-api-service-ca\") pod \"bc27ee59-8e16-4bf8-84ed-3ff4ebab192b\" (UID: \"bc27ee59-8e16-4bf8-84ed-3ff4ebab192b\") "
Sep 19 11:45:38.496386 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:38.496362    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc27ee59-8e16-4bf8-84ed-3ff4ebab192b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bc27ee59-8e16-4bf8-84ed-3ff4ebab192b" (UID: "bc27ee59-8e16-4bf8-84ed-3ff4ebab192b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:45:38.597137 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:38.597110    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc27ee59-8e16-4bf8-84ed-3ff4ebab192b-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:45:39.237119 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:39.237084    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5deb747f-8hp75"
Sep 19 11:45:39.237119 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:39.237107    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5deb747f-8hp75" event={"ID":"bc27ee59-8e16-4bf8-84ed-3ff4ebab192b","Type":"ContainerDied","Data":"f022eb669dc9c0c31f2d6b98d39c21dbaf1d6a9ad035a30d82fda1281645ebfd"}
Sep 19 11:45:39.237367 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:39.237158    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f022eb669dc9c0c31f2d6b98d39c21dbaf1d6a9ad035a30d82fda1281645ebfd"
Sep 19 11:45:41.887938 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:41.887894    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"]
Sep 19 11:45:41.888387 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:41.888160    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q" podUID="4984308b-4cbc-46d2-867e-df5e77e43a0a" containerName="payload-processing" containerID="cri-o://0ab00ea4319cf822e4ee8a62182552a73a35f0c1ad19917f1551b63dd141db7a" gracePeriod=30
Sep 19 11:45:42.044764 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.044729    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5deb747f-8hp75"]
Sep 19 11:45:42.047087 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.047054    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5deb747f-8hp75"]
Sep 19 11:45:42.163501 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.163473    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"
Sep 19 11:45:42.235017 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.234981    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9c72q\" (UniqueName: \"kubernetes.io/projected/4984308b-4cbc-46d2-867e-df5e77e43a0a-kube-api-access-9c72q\") pod \"4984308b-4cbc-46d2-867e-df5e77e43a0a\" (UID: \"4984308b-4cbc-46d2-867e-df5e77e43a0a\") "
Sep 19 11:45:42.235252 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.235104    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4984308b-4cbc-46d2-867e-df5e77e43a0a-plugins-config-volume\") pod \"4984308b-4cbc-46d2-867e-df5e77e43a0a\" (UID: \"4984308b-4cbc-46d2-867e-df5e77e43a0a\") "
Sep 19 11:45:42.235463 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.235437    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4984308b-4cbc-46d2-867e-df5e77e43a0a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4984308b-4cbc-46d2-867e-df5e77e43a0a" (UID: "4984308b-4cbc-46d2-867e-df5e77e43a0a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:45:42.237284 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.237263    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4984308b-4cbc-46d2-867e-df5e77e43a0a-kube-api-access-9c72q" (OuterVolumeSpecName: "kube-api-access-9c72q") pod "4984308b-4cbc-46d2-867e-df5e77e43a0a" (UID: "4984308b-4cbc-46d2-867e-df5e77e43a0a"). InnerVolumeSpecName "kube-api-access-9c72q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:45:42.254804 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.254776    2559 generic.go:356] "Generic (PLEG): container finished" podID="4984308b-4cbc-46d2-867e-df5e77e43a0a" containerID="0ab00ea4319cf822e4ee8a62182552a73a35f0c1ad19917f1551b63dd141db7a" exitCode=0
Sep 19 11:45:42.254971 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.254834    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"
Sep 19 11:45:42.254971 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.254864    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q" event={"ID":"4984308b-4cbc-46d2-867e-df5e77e43a0a","Type":"ContainerDied","Data":"0ab00ea4319cf822e4ee8a62182552a73a35f0c1ad19917f1551b63dd141db7a"}
Sep 19 11:45:42.254971 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.254917    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q" event={"ID":"4984308b-4cbc-46d2-867e-df5e77e43a0a","Type":"ContainerDied","Data":"7464a09888c09c1749445539b4e198cafe95a69eb811ee529301234bbbcb5771"}
Sep 19 11:45:42.254971 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.254942    2559 scope.go:117] "RemoveContainer" containerID="0ab00ea4319cf822e4ee8a62182552a73a35f0c1ad19917f1551b63dd141db7a"
Sep 19 11:45:42.266664 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.266643    2559 scope.go:117] "RemoveContainer" containerID="0ab00ea4319cf822e4ee8a62182552a73a35f0c1ad19917f1551b63dd141db7a"
Sep 19 11:45:42.266997 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:45:42.266972    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ab00ea4319cf822e4ee8a62182552a73a35f0c1ad19917f1551b63dd141db7a\": container with ID starting with 0ab00ea4319cf822e4ee8a62182552a73a35f0c1ad19917f1551b63dd141db7a not found: ID does not exist" containerID="0ab00ea4319cf822e4ee8a62182552a73a35f0c1ad19917f1551b63dd141db7a"
Sep 19 11:45:42.267088 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.267011    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ab00ea4319cf822e4ee8a62182552a73a35f0c1ad19917f1551b63dd141db7a"} err="failed to get container status \"0ab00ea4319cf822e4ee8a62182552a73a35f0c1ad19917f1551b63dd141db7a\": rpc error: code = NotFound desc = could not find container \"0ab00ea4319cf822e4ee8a62182552a73a35f0c1ad19917f1551b63dd141db7a\": container with ID starting with 0ab00ea4319cf822e4ee8a62182552a73a35f0c1ad19917f1551b63dd141db7a not found: ID does not exist"
Sep 19 11:45:42.281123 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.281076    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"]
Sep 19 11:45:42.284263 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.284239    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5deb747f-5874cf6c6c-cvp5q"]
Sep 19 11:45:42.336059 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.336014    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4984308b-4cbc-46d2-867e-df5e77e43a0a-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:45:42.336059 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.336055    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c72q\" (UniqueName: \"kubernetes.io/projected/4984308b-4cbc-46d2-867e-df5e77e43a0a-kube-api-access-9c72q\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:45:42.595771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.595741    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4984308b-4cbc-46d2-867e-df5e77e43a0a" path="/var/lib/kubelet/pods/4984308b-4cbc-46d2-867e-df5e77e43a0a/volumes"
Sep 19 11:45:42.596083 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:42.596070    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc27ee59-8e16-4bf8-84ed-3ff4ebab192b" path="/var/lib/kubelet/pods/bc27ee59-8e16-4bf8-84ed-3ff4ebab192b/volumes"
Sep 19 11:45:51.776679 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.776652    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"]
Sep 19 11:45:51.777109 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.777073    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4984308b-4cbc-46d2-867e-df5e77e43a0a" containerName="payload-processing"
Sep 19 11:45:51.777109 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.777084    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4984308b-4cbc-46d2-867e-df5e77e43a0a" containerName="payload-processing"
Sep 19 11:45:51.777109 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.777110    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc27ee59-8e16-4bf8-84ed-3ff4ebab192b" containerName="revoke-keys"
Sep 19 11:45:51.777252 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.777116    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc27ee59-8e16-4bf8-84ed-3ff4ebab192b" containerName="revoke-keys"
Sep 19 11:45:51.777252 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.777185    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc27ee59-8e16-4bf8-84ed-3ff4ebab192b" containerName="revoke-keys"
Sep 19 11:45:51.777252 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.777213    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4984308b-4cbc-46d2-867e-df5e77e43a0a" containerName="payload-processing"
Sep 19 11:45:51.787061 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.787031    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"
Sep 19 11:45:51.791581 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.791561    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-db360f2c-dockercfg-k9djz\""
Sep 19 11:45:51.791707 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.791568    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-db360f2c\""
Sep 19 11:45:51.805913 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.805890    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"]
Sep 19 11:45:51.826292 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.826251    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"]
Sep 19 11:45:51.830684 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.830665    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"
Sep 19 11:45:51.859051 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.859026    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"]
Sep 19 11:45:51.934681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.934655    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eaef035-d813-4265-9bc3-5cca61a3b5b9-plugins-config-volume\") pod \"payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr\" (UID: \"7eaef035-d813-4265-9bc3-5cca61a3b5b9\") " pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"
Sep 19 11:45:51.934681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.934685    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb25h\" (UniqueName: \"kubernetes.io/projected/7eaef035-d813-4265-9bc3-5cca61a3b5b9-kube-api-access-sb25h\") pod \"payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr\" (UID: \"7eaef035-d813-4265-9bc3-5cca61a3b5b9\") " pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"
Sep 19 11:45:51.934865 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.934775    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnc6p\" (UniqueName: \"kubernetes.io/projected/b0176f0f-4bfb-4cf0-8ef4-563df03ab014-kube-api-access-tnc6p\") pod \"payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh\" (UID: \"b0176f0f-4bfb-4cf0-8ef4-563df03ab014\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"
Sep 19 11:45:51.934865 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:51.934811    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0176f0f-4bfb-4cf0-8ef4-563df03ab014-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh\" (UID: \"b0176f0f-4bfb-4cf0-8ef4-563df03ab014\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"
Sep 19 11:45:52.035954 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:52.035878    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0176f0f-4bfb-4cf0-8ef4-563df03ab014-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh\" (UID: \"b0176f0f-4bfb-4cf0-8ef4-563df03ab014\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"
Sep 19 11:45:52.036100 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:52.035977    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eaef035-d813-4265-9bc3-5cca61a3b5b9-plugins-config-volume\") pod \"payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr\" (UID: \"7eaef035-d813-4265-9bc3-5cca61a3b5b9\") " pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"
Sep 19 11:45:52.036100 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:52.035996    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb25h\" (UniqueName: \"kubernetes.io/projected/7eaef035-d813-4265-9bc3-5cca61a3b5b9-kube-api-access-sb25h\") pod \"payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr\" (UID: \"7eaef035-d813-4265-9bc3-5cca61a3b5b9\") " pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"
Sep 19 11:45:52.036100 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:52.036050    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnc6p\" (UniqueName: \"kubernetes.io/projected/b0176f0f-4bfb-4cf0-8ef4-563df03ab014-kube-api-access-tnc6p\") pod \"payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh\" (UID: \"b0176f0f-4bfb-4cf0-8ef4-563df03ab014\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"
Sep 19 11:45:52.036729 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:52.036708    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0176f0f-4bfb-4cf0-8ef4-563df03ab014-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh\" (UID: \"b0176f0f-4bfb-4cf0-8ef4-563df03ab014\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"
Sep 19 11:45:52.036775 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:52.036714    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eaef035-d813-4265-9bc3-5cca61a3b5b9-plugins-config-volume\") pod \"payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr\" (UID: \"7eaef035-d813-4265-9bc3-5cca61a3b5b9\") " pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"
Sep 19 11:45:52.043656 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:52.043637    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnc6p\" (UniqueName: \"kubernetes.io/projected/b0176f0f-4bfb-4cf0-8ef4-563df03ab014-kube-api-access-tnc6p\") pod \"payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh\" (UID: \"b0176f0f-4bfb-4cf0-8ef4-563df03ab014\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"
Sep 19 11:45:52.043768 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:52.043724    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb25h\" (UniqueName: \"kubernetes.io/projected/7eaef035-d813-4265-9bc3-5cca61a3b5b9-kube-api-access-sb25h\") pod \"payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr\" (UID: \"7eaef035-d813-4265-9bc3-5cca61a3b5b9\") " pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"
Sep 19 11:45:52.097632 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:52.097606    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"
Sep 19 11:45:52.142037 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:52.142008    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"
Sep 19 11:45:52.370935 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:52.370910    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"]
Sep 19 11:45:52.371572 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:45:52.371552    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eaef035_d813_4265_9bc3_5cca61a3b5b9.slice/crio-123b3d327819498b84784da43615949cbd90e187763a682d705ad8de2cd131a6 WatchSource:0}: Error finding container 123b3d327819498b84784da43615949cbd90e187763a682d705ad8de2cd131a6: Status 404 returned error can't find the container with id 123b3d327819498b84784da43615949cbd90e187763a682d705ad8de2cd131a6
Sep 19 11:45:52.434972 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:52.434945    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"]
Sep 19 11:45:52.436753 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:45:52.436729    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0176f0f_4bfb_4cf0_8ef4_563df03ab014.slice/crio-af64bbb25bf52a6578a079855df11a651987e31bd495292612fa58f6dd137234 WatchSource:0}: Error finding container af64bbb25bf52a6578a079855df11a651987e31bd495292612fa58f6dd137234: Status 404 returned error can't find the container with id af64bbb25bf52a6578a079855df11a651987e31bd495292612fa58f6dd137234
Sep 19 11:45:53.321881 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:53.321846    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh" event={"ID":"b0176f0f-4bfb-4cf0-8ef4-563df03ab014","Type":"ContainerStarted","Data":"18a0b75f2e696f0d6e442b060b2ee9f867c14d106008eb80bf2d12b827247dda"}
Sep 19 11:45:53.322371 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:53.321906    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh" event={"ID":"b0176f0f-4bfb-4cf0-8ef4-563df03ab014","Type":"ContainerStarted","Data":"af64bbb25bf52a6578a079855df11a651987e31bd495292612fa58f6dd137234"}
Sep 19 11:45:53.322371 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:53.322019    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"
Sep 19 11:45:53.323911 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:53.323869    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr" event={"ID":"7eaef035-d813-4265-9bc3-5cca61a3b5b9","Type":"ContainerStarted","Data":"9fe11a9d02128443075b40f45afce8c10fab6650cf76b6d6792b36093e946d66"}
Sep 19 11:45:53.324024 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:53.323920    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr" event={"ID":"7eaef035-d813-4265-9bc3-5cca61a3b5b9","Type":"ContainerStarted","Data":"123b3d327819498b84784da43615949cbd90e187763a682d705ad8de2cd131a6"}
Sep 19 11:45:53.324024 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:53.323949    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"
Sep 19 11:45:53.339362 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:53.339313    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh" podStartSLOduration=2.339300625 podStartE2EDuration="2.339300625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:45:53.33678301 +0000 UTC m=+705.259095894" watchObservedRunningTime="2026-09-19 11:45:53.339300625 +0000 UTC m=+705.261613505"
Sep 19 11:45:53.352476 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:45:53.352434    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr" podStartSLOduration=2.352422153 podStartE2EDuration="2.352422153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:45:53.350301825 +0000 UTC m=+705.272614707" watchObservedRunningTime="2026-09-19 11:45:53.352422153 +0000 UTC m=+705.274735034"
Sep 19 11:46:04.332910 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:04.332878    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"
Sep 19 11:46:04.333301 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:04.332945    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"
Sep 19 11:46:16.834470 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:16.834435    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5"]
Sep 19 11:46:16.839395 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:16.839378    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5"
Sep 19 11:46:16.842693 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:16.842672    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:46:16.844956 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:16.844936    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5"]
Sep 19 11:46:16.866250 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:16.866230    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e518aed-f62a-46f4-a464-6fe78ecf3eaf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5\" (UID: \"9e518aed-f62a-46f4-a464-6fe78ecf3eaf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5"
Sep 19 11:46:16.967491 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:16.967462    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e518aed-f62a-46f4-a464-6fe78ecf3eaf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5\" (UID: \"9e518aed-f62a-46f4-a464-6fe78ecf3eaf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5"
Sep 19 11:46:16.968029 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:16.968010    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e518aed-f62a-46f4-a464-6fe78ecf3eaf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5\" (UID: \"9e518aed-f62a-46f4-a464-6fe78ecf3eaf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5"
Sep 19 11:46:17.152275 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:17.152179    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5"
Sep 19 11:46:17.424310 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:17.424281    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5"]
Sep 19 11:46:17.426015 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:46:17.425991    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e518aed_f62a_46f4_a464_6fe78ecf3eaf.slice/crio-951c108fe9c5ecab2d9a332836bff3f76efe759f515a10ce97ebe9c32bac60e4 WatchSource:0}: Error finding container 951c108fe9c5ecab2d9a332836bff3f76efe759f515a10ce97ebe9c32bac60e4: Status 404 returned error can't find the container with id 951c108fe9c5ecab2d9a332836bff3f76efe759f515a10ce97ebe9c32bac60e4
Sep 19 11:46:17.468525 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:17.468497    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5" event={"ID":"9e518aed-f62a-46f4-a464-6fe78ecf3eaf","Type":"ContainerStarted","Data":"951c108fe9c5ecab2d9a332836bff3f76efe759f515a10ce97ebe9c32bac60e4"}
Sep 19 11:46:18.475810 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:18.475779    2559 generic.go:356] "Generic (PLEG): container finished" podID="9e518aed-f62a-46f4-a464-6fe78ecf3eaf" containerID="56a1b52a100459632d881e86f9d8ff8c1f1f9d637db2f368248818efe4dcd588" exitCode=0
Sep 19 11:46:18.476228 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:18.475862    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5" event={"ID":"9e518aed-f62a-46f4-a464-6fe78ecf3eaf","Type":"ContainerDied","Data":"56a1b52a100459632d881e86f9d8ff8c1f1f9d637db2f368248818efe4dcd588"}
Sep 19 11:46:19.625274 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:19.625249    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5"
Sep 19 11:46:19.697220 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:19.697156    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e518aed-f62a-46f4-a464-6fe78ecf3eaf-maas-api-service-ca\") pod \"9e518aed-f62a-46f4-a464-6fe78ecf3eaf\" (UID: \"9e518aed-f62a-46f4-a464-6fe78ecf3eaf\") "
Sep 19 11:46:19.697543 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:19.697521    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e518aed-f62a-46f4-a464-6fe78ecf3eaf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e518aed-f62a-46f4-a464-6fe78ecf3eaf" (UID: "9e518aed-f62a-46f4-a464-6fe78ecf3eaf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:46:19.798186 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:19.798096    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e518aed-f62a-46f4-a464-6fe78ecf3eaf-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:46:20.490700 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:20.490661    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5" event={"ID":"9e518aed-f62a-46f4-a464-6fe78ecf3eaf","Type":"ContainerDied","Data":"951c108fe9c5ecab2d9a332836bff3f76efe759f515a10ce97ebe9c32bac60e4"}
Sep 19 11:46:20.490700 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:20.490690    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5"
Sep 19 11:46:20.490908 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:20.490714    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="951c108fe9c5ecab2d9a332836bff3f76efe759f515a10ce97ebe9c32bac60e4"
Sep 19 11:46:25.969934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:25.969899    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-7a524f-b2bpr"]
Sep 19 11:46:25.970412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:25.970382    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e518aed-f62a-46f4-a464-6fe78ecf3eaf" containerName="revoke-keys"
Sep 19 11:46:25.970412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:25.970394    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e518aed-f62a-46f4-a464-6fe78ecf3eaf" containerName="revoke-keys"
Sep 19 11:46:25.970486 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:25.970476    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e518aed-f62a-46f4-a464-6fe78ecf3eaf" containerName="revoke-keys"
Sep 19 11:46:25.974006 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:25.973990    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-7a524f-b2bpr"
Sep 19 11:46:25.976140 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:25.976122    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:46:25.980773 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:25.980745    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-7a524f-b2bpr"]
Sep 19 11:46:26.060564 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:26.060521    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bd08d4-6112-453a-b7ce-37ae4d18342f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-7a524f-b2bpr\" (UID: \"53bd08d4-6112-453a-b7ce-37ae4d18342f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-7a524f-b2bpr"
Sep 19 11:46:26.161488 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:26.161444    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bd08d4-6112-453a-b7ce-37ae4d18342f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-7a524f-b2bpr\" (UID: \"53bd08d4-6112-453a-b7ce-37ae4d18342f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-7a524f-b2bpr"
Sep 19 11:46:26.162057 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:26.162034    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bd08d4-6112-453a-b7ce-37ae4d18342f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-7a524f-b2bpr\" (UID: \"53bd08d4-6112-453a-b7ce-37ae4d18342f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-7a524f-b2bpr"
Sep 19 11:46:26.287699 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:26.287597    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-7a524f-b2bpr"
Sep 19 11:46:26.560511 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:26.560487    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-7a524f-b2bpr"]
Sep 19 11:46:26.562605 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:46:26.562575    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53bd08d4_6112_453a_b7ce_37ae4d18342f.slice/crio-f6b714d0453d585affbbfaf71a4e64701291c9e38e4d7576ac5340b2681519d0 WatchSource:0}: Error finding container f6b714d0453d585affbbfaf71a4e64701291c9e38e4d7576ac5340b2681519d0: Status 404 returned error can't find the container with id f6b714d0453d585affbbfaf71a4e64701291c9e38e4d7576ac5340b2681519d0
Sep 19 11:46:27.533798 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:27.533764    2559 generic.go:356] "Generic (PLEG): container finished" podID="53bd08d4-6112-453a-b7ce-37ae4d18342f" containerID="19a3bc453bf3dfa21d80a0a1b6a6dd4e5c2a494ed94d9b756d2ad101066a4d5c" exitCode=0
Sep 19 11:46:27.534227 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:27.533848    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-7a524f-b2bpr" event={"ID":"53bd08d4-6112-453a-b7ce-37ae4d18342f","Type":"ContainerDied","Data":"19a3bc453bf3dfa21d80a0a1b6a6dd4e5c2a494ed94d9b756d2ad101066a4d5c"}
Sep 19 11:46:27.534227 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:27.533914    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-7a524f-b2bpr" event={"ID":"53bd08d4-6112-453a-b7ce-37ae4d18342f","Type":"ContainerStarted","Data":"f6b714d0453d585affbbfaf71a4e64701291c9e38e4d7576ac5340b2681519d0"}
Sep 19 11:46:28.677783 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:28.677757    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-7a524f-b2bpr"
Sep 19 11:46:28.790471 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:28.790383    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bd08d4-6112-453a-b7ce-37ae4d18342f-maas-api-service-ca\") pod \"53bd08d4-6112-453a-b7ce-37ae4d18342f\" (UID: \"53bd08d4-6112-453a-b7ce-37ae4d18342f\") "
Sep 19 11:46:28.790698 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:28.790673    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53bd08d4-6112-453a-b7ce-37ae4d18342f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "53bd08d4-6112-453a-b7ce-37ae4d18342f" (UID: "53bd08d4-6112-453a-b7ce-37ae4d18342f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:46:28.790788 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:28.790773    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bd08d4-6112-453a-b7ce-37ae4d18342f-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:46:29.376266 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.376231    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"]
Sep 19 11:46:29.376586 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.376531    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr" podUID="7eaef035-d813-4265-9bc3-5cca61a3b5b9" containerName="payload-processing" containerID="cri-o://9fe11a9d02128443075b40f45afce8c10fab6650cf76b6d6792b36093e946d66" gracePeriod=30
Sep 19 11:46:29.384577 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.384549    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"]
Sep 19 11:46:29.385098 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.385069    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh" podUID="b0176f0f-4bfb-4cf0-8ef4-563df03ab014" containerName="payload-pre-processing" containerID="cri-o://18a0b75f2e696f0d6e442b060b2ee9f867c14d106008eb80bf2d12b827247dda" gracePeriod=30
Sep 19 11:46:29.551688 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.551651    2559 generic.go:356] "Generic (PLEG): container finished" podID="b0176f0f-4bfb-4cf0-8ef4-563df03ab014" containerID="18a0b75f2e696f0d6e442b060b2ee9f867c14d106008eb80bf2d12b827247dda" exitCode=0
Sep 19 11:46:29.551856 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.551719    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh" event={"ID":"b0176f0f-4bfb-4cf0-8ef4-563df03ab014","Type":"ContainerDied","Data":"18a0b75f2e696f0d6e442b060b2ee9f867c14d106008eb80bf2d12b827247dda"}
Sep 19 11:46:29.553819 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.553788    2559 generic.go:356] "Generic (PLEG): container finished" podID="7eaef035-d813-4265-9bc3-5cca61a3b5b9" containerID="9fe11a9d02128443075b40f45afce8c10fab6650cf76b6d6792b36093e946d66" exitCode=0
Sep 19 11:46:29.553974 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.553870    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr" event={"ID":"7eaef035-d813-4265-9bc3-5cca61a3b5b9","Type":"ContainerDied","Data":"9fe11a9d02128443075b40f45afce8c10fab6650cf76b6d6792b36093e946d66"}
Sep 19 11:46:29.555821 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.555493    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-7a524f-b2bpr" event={"ID":"53bd08d4-6112-453a-b7ce-37ae4d18342f","Type":"ContainerDied","Data":"f6b714d0453d585affbbfaf71a4e64701291c9e38e4d7576ac5340b2681519d0"}
Sep 19 11:46:29.555821 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.555531    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6b714d0453d585affbbfaf71a4e64701291c9e38e4d7576ac5340b2681519d0"
Sep 19 11:46:29.555821 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.555588    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-7a524f-b2bpr"
Sep 19 11:46:29.649421 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.649397    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"
Sep 19 11:46:29.686784 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.686759    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"
Sep 19 11:46:29.715755 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.715727    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5"]
Sep 19 11:46:29.719568 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.719445    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-db360f2c-8wpp5"]
Sep 19 11:46:29.800396 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.800366    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnc6p\" (UniqueName: \"kubernetes.io/projected/b0176f0f-4bfb-4cf0-8ef4-563df03ab014-kube-api-access-tnc6p\") pod \"b0176f0f-4bfb-4cf0-8ef4-563df03ab014\" (UID: \"b0176f0f-4bfb-4cf0-8ef4-563df03ab014\") "
Sep 19 11:46:29.800596 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.800410    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eaef035-d813-4265-9bc3-5cca61a3b5b9-plugins-config-volume\") pod \"7eaef035-d813-4265-9bc3-5cca61a3b5b9\" (UID: \"7eaef035-d813-4265-9bc3-5cca61a3b5b9\") "
Sep 19 11:46:29.800596 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.800584    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0176f0f-4bfb-4cf0-8ef4-563df03ab014-plugins-config-volume\") pod \"b0176f0f-4bfb-4cf0-8ef4-563df03ab014\" (UID: \"b0176f0f-4bfb-4cf0-8ef4-563df03ab014\") "
Sep 19 11:46:29.800717 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.800623    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sb25h\" (UniqueName: \"kubernetes.io/projected/7eaef035-d813-4265-9bc3-5cca61a3b5b9-kube-api-access-sb25h\") pod \"7eaef035-d813-4265-9bc3-5cca61a3b5b9\" (UID: \"7eaef035-d813-4265-9bc3-5cca61a3b5b9\") "
Sep 19 11:46:29.800933 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.800907    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7eaef035-d813-4265-9bc3-5cca61a3b5b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7eaef035-d813-4265-9bc3-5cca61a3b5b9" (UID: "7eaef035-d813-4265-9bc3-5cca61a3b5b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:46:29.800933 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.800924    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0176f0f-4bfb-4cf0-8ef4-563df03ab014-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b0176f0f-4bfb-4cf0-8ef4-563df03ab014" (UID: "b0176f0f-4bfb-4cf0-8ef4-563df03ab014"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:46:29.802706 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.802679    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eaef035-d813-4265-9bc3-5cca61a3b5b9-kube-api-access-sb25h" (OuterVolumeSpecName: "kube-api-access-sb25h") pod "7eaef035-d813-4265-9bc3-5cca61a3b5b9" (UID: "7eaef035-d813-4265-9bc3-5cca61a3b5b9"). InnerVolumeSpecName "kube-api-access-sb25h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:46:29.802770 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.802727    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0176f0f-4bfb-4cf0-8ef4-563df03ab014-kube-api-access-tnc6p" (OuterVolumeSpecName: "kube-api-access-tnc6p") pod "b0176f0f-4bfb-4cf0-8ef4-563df03ab014" (UID: "b0176f0f-4bfb-4cf0-8ef4-563df03ab014"). InnerVolumeSpecName "kube-api-access-tnc6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:46:29.902318 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.902226    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0176f0f-4bfb-4cf0-8ef4-563df03ab014-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:46:29.902318 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.902257    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sb25h\" (UniqueName: \"kubernetes.io/projected/7eaef035-d813-4265-9bc3-5cca61a3b5b9-kube-api-access-sb25h\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:46:29.902318 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.902270    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnc6p\" (UniqueName: \"kubernetes.io/projected/b0176f0f-4bfb-4cf0-8ef4-563df03ab014-kube-api-access-tnc6p\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:46:29.902318 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:29.902281    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eaef035-d813-4265-9bc3-5cca61a3b5b9-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:46:30.563233 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:30.563180    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"
Sep 19 11:46:30.563233 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:30.563207    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh" event={"ID":"b0176f0f-4bfb-4cf0-8ef4-563df03ab014","Type":"ContainerDied","Data":"af64bbb25bf52a6578a079855df11a651987e31bd495292612fa58f6dd137234"}
Sep 19 11:46:30.563466 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:30.563276    2559 scope.go:117] "RemoveContainer" containerID="18a0b75f2e696f0d6e442b060b2ee9f867c14d106008eb80bf2d12b827247dda"
Sep 19 11:46:30.564699 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:30.564668    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr" event={"ID":"7eaef035-d813-4265-9bc3-5cca61a3b5b9","Type":"ContainerDied","Data":"123b3d327819498b84784da43615949cbd90e187763a682d705ad8de2cd131a6"}
Sep 19 11:46:30.564699 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:30.564698    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"
Sep 19 11:46:30.574983 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:30.574968    2559 scope.go:117] "RemoveContainer" containerID="9fe11a9d02128443075b40f45afce8c10fab6650cf76b6d6792b36093e946d66"
Sep 19 11:46:30.597472 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:30.597434    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e518aed-f62a-46f4-a464-6fe78ecf3eaf" path="/var/lib/kubelet/pods/9e518aed-f62a-46f4-a464-6fe78ecf3eaf/volumes"
Sep 19 11:46:30.597835 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:30.597818    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"]
Sep 19 11:46:30.597876 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:30.597854    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-db360f2c-578b9456fd-27jrh"]
Sep 19 11:46:30.616847 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:30.616806    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"]
Sep 19 11:46:30.619037 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:30.619016    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-db360f2c-587b5f67c7-mm2fr"]
Sep 19 11:46:32.595356 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:32.595322    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eaef035-d813-4265-9bc3-5cca61a3b5b9" path="/var/lib/kubelet/pods/7eaef035-d813-4265-9bc3-5cca61a3b5b9/volumes"
Sep 19 11:46:32.595731 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:32.595676    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0176f0f-4bfb-4cf0-8ef4-563df03ab014" path="/var/lib/kubelet/pods/b0176f0f-4bfb-4cf0-8ef4-563df03ab014/volumes"
Sep 19 11:46:36.958144 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:36.958107    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"]
Sep 19 11:46:36.958645 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:36.958372    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll" podUID="0541c39e-ae1a-4231-8f83-b59f2dcff552" containerName="payload-pre-processing" containerID="cri-o://969cd528ccb687178558ad2f2564be1b674181ed0f3d54e51200da601bde0d99" gracePeriod=30
Sep 19 11:46:37.106974 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.106893    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-7a524f-b2bpr"]
Sep 19 11:46:37.111267 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.111239    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-7a524f-b2bpr"]
Sep 19 11:46:37.235095 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.235060    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"
Sep 19 11:46:37.388866 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.388834    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0541c39e-ae1a-4231-8f83-b59f2dcff552-plugins-config-volume\") pod \"0541c39e-ae1a-4231-8f83-b59f2dcff552\" (UID: \"0541c39e-ae1a-4231-8f83-b59f2dcff552\") "
Sep 19 11:46:37.389075 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.388893    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf44c\" (UniqueName: \"kubernetes.io/projected/0541c39e-ae1a-4231-8f83-b59f2dcff552-kube-api-access-mf44c\") pod \"0541c39e-ae1a-4231-8f83-b59f2dcff552\" (UID: \"0541c39e-ae1a-4231-8f83-b59f2dcff552\") "
Sep 19 11:46:37.389254 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.389232    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0541c39e-ae1a-4231-8f83-b59f2dcff552-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0541c39e-ae1a-4231-8f83-b59f2dcff552" (UID: "0541c39e-ae1a-4231-8f83-b59f2dcff552"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:46:37.391118 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.391091    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0541c39e-ae1a-4231-8f83-b59f2dcff552-kube-api-access-mf44c" (OuterVolumeSpecName: "kube-api-access-mf44c") pod "0541c39e-ae1a-4231-8f83-b59f2dcff552" (UID: "0541c39e-ae1a-4231-8f83-b59f2dcff552"). InnerVolumeSpecName "kube-api-access-mf44c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:46:37.490315 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.490231    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0541c39e-ae1a-4231-8f83-b59f2dcff552-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:46:37.490315 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.490264    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf44c\" (UniqueName: \"kubernetes.io/projected/0541c39e-ae1a-4231-8f83-b59f2dcff552-kube-api-access-mf44c\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:46:37.609667 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.609625    2559 generic.go:356] "Generic (PLEG): container finished" podID="0541c39e-ae1a-4231-8f83-b59f2dcff552" containerID="969cd528ccb687178558ad2f2564be1b674181ed0f3d54e51200da601bde0d99" exitCode=0
Sep 19 11:46:37.609843 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.609685    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"
Sep 19 11:46:37.609843 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.609709    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll" event={"ID":"0541c39e-ae1a-4231-8f83-b59f2dcff552","Type":"ContainerDied","Data":"969cd528ccb687178558ad2f2564be1b674181ed0f3d54e51200da601bde0d99"}
Sep 19 11:46:37.609843 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.609761    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll" event={"ID":"0541c39e-ae1a-4231-8f83-b59f2dcff552","Type":"ContainerDied","Data":"d7e4eb647e2f7c77b409254d2ffb026f26a6298ad59d1076fc4dc1a8dfce8247"}
Sep 19 11:46:37.609843 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.609785    2559 scope.go:117] "RemoveContainer" containerID="969cd528ccb687178558ad2f2564be1b674181ed0f3d54e51200da601bde0d99"
Sep 19 11:46:37.625642 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.625622    2559 scope.go:117] "RemoveContainer" containerID="969cd528ccb687178558ad2f2564be1b674181ed0f3d54e51200da601bde0d99"
Sep 19 11:46:37.625975 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:46:37.625957    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"969cd528ccb687178558ad2f2564be1b674181ed0f3d54e51200da601bde0d99\": container with ID starting with 969cd528ccb687178558ad2f2564be1b674181ed0f3d54e51200da601bde0d99 not found: ID does not exist" containerID="969cd528ccb687178558ad2f2564be1b674181ed0f3d54e51200da601bde0d99"
Sep 19 11:46:37.626027 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.625985    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"969cd528ccb687178558ad2f2564be1b674181ed0f3d54e51200da601bde0d99"} err="failed to get container status \"969cd528ccb687178558ad2f2564be1b674181ed0f3d54e51200da601bde0d99\": rpc error: code = NotFound desc = could not find container \"969cd528ccb687178558ad2f2564be1b674181ed0f3d54e51200da601bde0d99\": container with ID starting with 969cd528ccb687178558ad2f2564be1b674181ed0f3d54e51200da601bde0d99 not found: ID does not exist"
Sep 19 11:46:37.634164 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.634134    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"]
Sep 19 11:46:37.641930 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:37.641905    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-7a524f-6b7964f96b-dk4ll"]
Sep 19 11:46:38.596146 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:38.596116    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0541c39e-ae1a-4231-8f83-b59f2dcff552" path="/var/lib/kubelet/pods/0541c39e-ae1a-4231-8f83-b59f2dcff552/volumes"
Sep 19 11:46:38.596519 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:46:38.596478    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53bd08d4-6112-453a-b7ce-37ae4d18342f" path="/var/lib/kubelet/pods/53bd08d4-6112-453a-b7ce-37ae4d18342f/volumes"
Sep 19 11:47:10.671153 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.671119    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d"]
Sep 19 11:47:10.671589 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.671575    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0176f0f-4bfb-4cf0-8ef4-563df03ab014" containerName="payload-pre-processing"
Sep 19 11:47:10.671632 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.671590    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0176f0f-4bfb-4cf0-8ef4-563df03ab014" containerName="payload-pre-processing"
Sep 19 11:47:10.671632 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.671608    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53bd08d4-6112-453a-b7ce-37ae4d18342f" containerName="revoke-keys"
Sep 19 11:47:10.671632 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.671613    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="53bd08d4-6112-453a-b7ce-37ae4d18342f" containerName="revoke-keys"
Sep 19 11:47:10.671632 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.671624    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0541c39e-ae1a-4231-8f83-b59f2dcff552" containerName="payload-pre-processing"
Sep 19 11:47:10.671632 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.671631    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0541c39e-ae1a-4231-8f83-b59f2dcff552" containerName="payload-pre-processing"
Sep 19 11:47:10.671790 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.671637    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eaef035-d813-4265-9bc3-5cca61a3b5b9" containerName="payload-processing"
Sep 19 11:47:10.671790 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.671643    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eaef035-d813-4265-9bc3-5cca61a3b5b9" containerName="payload-processing"
Sep 19 11:47:10.671790 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.671711    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0541c39e-ae1a-4231-8f83-b59f2dcff552" containerName="payload-pre-processing"
Sep 19 11:47:10.671790 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.671720    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="53bd08d4-6112-453a-b7ce-37ae4d18342f" containerName="revoke-keys"
Sep 19 11:47:10.671790 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.671727    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0176f0f-4bfb-4cf0-8ef4-563df03ab014" containerName="payload-pre-processing"
Sep 19 11:47:10.671790 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.671735    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eaef035-d813-4265-9bc3-5cca61a3b5b9" containerName="payload-processing"
Sep 19 11:47:10.676085 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.676068    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d"
Sep 19 11:47:10.678607 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.678570    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:47:10.681542 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.681515    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d"]
Sep 19 11:47:10.808452 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.808418    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd21fff4-04ba-4e79-b7d6-200d906c5ea5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-37daed-j659d\" (UID: \"fd21fff4-04ba-4e79-b7d6-200d906c5ea5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d"
Sep 19 11:47:10.909717 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.909682    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd21fff4-04ba-4e79-b7d6-200d906c5ea5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-37daed-j659d\" (UID: \"fd21fff4-04ba-4e79-b7d6-200d906c5ea5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d"
Sep 19 11:47:10.910299 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.910282    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd21fff4-04ba-4e79-b7d6-200d906c5ea5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-37daed-j659d\" (UID: \"fd21fff4-04ba-4e79-b7d6-200d906c5ea5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d"
Sep 19 11:47:10.989046 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:10.989020    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d"
Sep 19 11:47:11.258506 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:11.258478    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d"]
Sep 19 11:47:11.260032 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:47:11.260006    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd21fff4_04ba_4e79_b7d6_200d906c5ea5.slice/crio-42a1f01607fee9cb0dad7d31bd27145fed59441d390378e73dcafc0a45612018 WatchSource:0}: Error finding container 42a1f01607fee9cb0dad7d31bd27145fed59441d390378e73dcafc0a45612018: Status 404 returned error can't find the container with id 42a1f01607fee9cb0dad7d31bd27145fed59441d390378e73dcafc0a45612018
Sep 19 11:47:11.808740 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:11.808705    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d" event={"ID":"fd21fff4-04ba-4e79-b7d6-200d906c5ea5","Type":"ContainerStarted","Data":"315014af181ea358db643119bce40b5f02f6902275f1c74598372394ca7b95a6"}
Sep 19 11:47:11.809232 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:11.808762    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d" event={"ID":"fd21fff4-04ba-4e79-b7d6-200d906c5ea5","Type":"ContainerStarted","Data":"42a1f01607fee9cb0dad7d31bd27145fed59441d390378e73dcafc0a45612018"}
Sep 19 11:47:11.827152 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:11.827093    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d" podStartSLOduration=1.827078067 podStartE2EDuration="1.827078067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:47:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:47:11.824357696 +0000 UTC m=+783.746670581" watchObservedRunningTime="2026-09-19 11:47:11.827078067 +0000 UTC m=+783.749390947"
Sep 19 11:47:12.815880 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:12.815849    2559 generic.go:356] "Generic (PLEG): container finished" podID="fd21fff4-04ba-4e79-b7d6-200d906c5ea5" containerID="315014af181ea358db643119bce40b5f02f6902275f1c74598372394ca7b95a6" exitCode=0
Sep 19 11:47:12.816278 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:12.815937    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d" event={"ID":"fd21fff4-04ba-4e79-b7d6-200d906c5ea5","Type":"ContainerDied","Data":"315014af181ea358db643119bce40b5f02f6902275f1c74598372394ca7b95a6"}
Sep 19 11:47:13.970842 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:13.970821    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d"
Sep 19 11:47:14.142775 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:14.142689    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd21fff4-04ba-4e79-b7d6-200d906c5ea5-maas-api-service-ca\") pod \"fd21fff4-04ba-4e79-b7d6-200d906c5ea5\" (UID: \"fd21fff4-04ba-4e79-b7d6-200d906c5ea5\") "
Sep 19 11:47:14.143062 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:14.143038    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd21fff4-04ba-4e79-b7d6-200d906c5ea5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fd21fff4-04ba-4e79-b7d6-200d906c5ea5" (UID: "fd21fff4-04ba-4e79-b7d6-200d906c5ea5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:47:14.244207 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:14.244167    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd21fff4-04ba-4e79-b7d6-200d906c5ea5-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:47:14.830926 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:14.830897    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d"
Sep 19 11:47:14.831127 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:14.830928    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d" event={"ID":"fd21fff4-04ba-4e79-b7d6-200d906c5ea5","Type":"ContainerDied","Data":"42a1f01607fee9cb0dad7d31bd27145fed59441d390378e73dcafc0a45612018"}
Sep 19 11:47:14.831127 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:14.830981    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="42a1f01607fee9cb0dad7d31bd27145fed59441d390378e73dcafc0a45612018"
Sep 19 11:47:21.159956 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.159920    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"]
Sep 19 11:47:21.160409 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.160168    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w" podUID="9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9" containerName="payload-processing" containerID="cri-o://0dc0fd588295d9a051215c0ca3a09b6586227b9216d427313458d6c4d2cb4185" gracePeriod=30
Sep 19 11:47:21.175292 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.175246    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"]
Sep 19 11:47:21.175536 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.175507    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx" podUID="8ea9fc31-ab89-4382-b201-7d0640395506" containerName="payload-pre-processing" containerID="cri-o://32e5ebb2921c67607abeba62dc2ba7837baba9eaf713b60021df8cac1cf37b4f" gracePeriod=30
Sep 19 11:47:21.327263 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.327226    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d"]
Sep 19 11:47:21.331174 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.331145    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-37daed-j659d"]
Sep 19 11:47:21.467544 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.467523    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"
Sep 19 11:47:21.470570 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.470549    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"
Sep 19 11:47:21.514101 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.514069    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9flgn\" (UniqueName: \"kubernetes.io/projected/9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9-kube-api-access-9flgn\") pod \"9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9\" (UID: \"9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9\") "
Sep 19 11:47:21.514277 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.514115    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ea9fc31-ab89-4382-b201-7d0640395506-plugins-config-volume\") pod \"8ea9fc31-ab89-4382-b201-7d0640395506\" (UID: \"8ea9fc31-ab89-4382-b201-7d0640395506\") "
Sep 19 11:47:21.514277 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.514154    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9-plugins-config-volume\") pod \"9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9\" (UID: \"9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9\") "
Sep 19 11:47:21.514374 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.514359    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpgdn\" (UniqueName: \"kubernetes.io/projected/8ea9fc31-ab89-4382-b201-7d0640395506-kube-api-access-rpgdn\") pod \"8ea9fc31-ab89-4382-b201-7d0640395506\" (UID: \"8ea9fc31-ab89-4382-b201-7d0640395506\") "
Sep 19 11:47:21.514531 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.514500    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9" (UID: "9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:47:21.514615 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.514515    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ea9fc31-ab89-4382-b201-7d0640395506-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8ea9fc31-ab89-4382-b201-7d0640395506" (UID: "8ea9fc31-ab89-4382-b201-7d0640395506"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:47:21.514705 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.514686    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ea9fc31-ab89-4382-b201-7d0640395506-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:47:21.514750 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.514710    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:47:21.516455 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.516430    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9-kube-api-access-9flgn" (OuterVolumeSpecName: "kube-api-access-9flgn") pod "9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9" (UID: "9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9"). InnerVolumeSpecName "kube-api-access-9flgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:47:21.516694 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.516676    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ea9fc31-ab89-4382-b201-7d0640395506-kube-api-access-rpgdn" (OuterVolumeSpecName: "kube-api-access-rpgdn") pod "8ea9fc31-ab89-4382-b201-7d0640395506" (UID: "8ea9fc31-ab89-4382-b201-7d0640395506"). InnerVolumeSpecName "kube-api-access-rpgdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:47:21.615161 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.615133    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpgdn\" (UniqueName: \"kubernetes.io/projected/8ea9fc31-ab89-4382-b201-7d0640395506-kube-api-access-rpgdn\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:47:21.615161 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.615157    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9flgn\" (UniqueName: \"kubernetes.io/projected/9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9-kube-api-access-9flgn\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:47:21.872778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.872686    2559 generic.go:356] "Generic (PLEG): container finished" podID="9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9" containerID="0dc0fd588295d9a051215c0ca3a09b6586227b9216d427313458d6c4d2cb4185" exitCode=0
Sep 19 11:47:21.872778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.872729    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"
Sep 19 11:47:21.872778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.872761    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w" event={"ID":"9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9","Type":"ContainerDied","Data":"0dc0fd588295d9a051215c0ca3a09b6586227b9216d427313458d6c4d2cb4185"}
Sep 19 11:47:21.873050 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.872822    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w" event={"ID":"9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9","Type":"ContainerDied","Data":"8fa625b2bf758ee0f13da57be181cae17b555a9c50510c9cc38d237f93656336"}
Sep 19 11:47:21.873050 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.872853    2559 scope.go:117] "RemoveContainer" containerID="0dc0fd588295d9a051215c0ca3a09b6586227b9216d427313458d6c4d2cb4185"
Sep 19 11:47:21.874434 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.874407    2559 generic.go:356] "Generic (PLEG): container finished" podID="8ea9fc31-ab89-4382-b201-7d0640395506" containerID="32e5ebb2921c67607abeba62dc2ba7837baba9eaf713b60021df8cac1cf37b4f" exitCode=0
Sep 19 11:47:21.874557 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.874460    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"
Sep 19 11:47:21.874557 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.874456    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx" event={"ID":"8ea9fc31-ab89-4382-b201-7d0640395506","Type":"ContainerDied","Data":"32e5ebb2921c67607abeba62dc2ba7837baba9eaf713b60021df8cac1cf37b4f"}
Sep 19 11:47:21.874692 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.874562    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx" event={"ID":"8ea9fc31-ab89-4382-b201-7d0640395506","Type":"ContainerDied","Data":"60a226794e901f3b094711387627549b85340bb49460373de77652fb7a6020fc"}
Sep 19 11:47:21.885536 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.885320    2559 scope.go:117] "RemoveContainer" containerID="0dc0fd588295d9a051215c0ca3a09b6586227b9216d427313458d6c4d2cb4185"
Sep 19 11:47:21.885837 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:47:21.885803    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dc0fd588295d9a051215c0ca3a09b6586227b9216d427313458d6c4d2cb4185\": container with ID starting with 0dc0fd588295d9a051215c0ca3a09b6586227b9216d427313458d6c4d2cb4185 not found: ID does not exist" containerID="0dc0fd588295d9a051215c0ca3a09b6586227b9216d427313458d6c4d2cb4185"
Sep 19 11:47:21.885939 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.885843    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dc0fd588295d9a051215c0ca3a09b6586227b9216d427313458d6c4d2cb4185"} err="failed to get container status \"0dc0fd588295d9a051215c0ca3a09b6586227b9216d427313458d6c4d2cb4185\": rpc error: code = NotFound desc = could not find container \"0dc0fd588295d9a051215c0ca3a09b6586227b9216d427313458d6c4d2cb4185\": container with ID starting with 0dc0fd588295d9a051215c0ca3a09b6586227b9216d427313458d6c4d2cb4185 not found: ID does not exist"
Sep 19 11:47:21.885939 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.885861    2559 scope.go:117] "RemoveContainer" containerID="32e5ebb2921c67607abeba62dc2ba7837baba9eaf713b60021df8cac1cf37b4f"
Sep 19 11:47:21.894860 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.894843    2559 scope.go:117] "RemoveContainer" containerID="32e5ebb2921c67607abeba62dc2ba7837baba9eaf713b60021df8cac1cf37b4f"
Sep 19 11:47:21.895130 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:47:21.895116    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32e5ebb2921c67607abeba62dc2ba7837baba9eaf713b60021df8cac1cf37b4f\": container with ID starting with 32e5ebb2921c67607abeba62dc2ba7837baba9eaf713b60021df8cac1cf37b4f not found: ID does not exist" containerID="32e5ebb2921c67607abeba62dc2ba7837baba9eaf713b60021df8cac1cf37b4f"
Sep 19 11:47:21.895171 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.895139    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32e5ebb2921c67607abeba62dc2ba7837baba9eaf713b60021df8cac1cf37b4f"} err="failed to get container status \"32e5ebb2921c67607abeba62dc2ba7837baba9eaf713b60021df8cac1cf37b4f\": rpc error: code = NotFound desc = could not find container \"32e5ebb2921c67607abeba62dc2ba7837baba9eaf713b60021df8cac1cf37b4f\": container with ID starting with 32e5ebb2921c67607abeba62dc2ba7837baba9eaf713b60021df8cac1cf37b4f not found: ID does not exist"
Sep 19 11:47:21.895232 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.895215    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"]
Sep 19 11:47:21.898344 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.898321    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-37daed-65874578c9-pcl2w"]
Sep 19 11:47:21.908669 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.908650    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"]
Sep 19 11:47:21.912455 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:21.912437    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-37daed-54d6f695db-td5tx"]
Sep 19 11:47:22.595172 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:22.595136    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ea9fc31-ab89-4382-b201-7d0640395506" path="/var/lib/kubelet/pods/8ea9fc31-ab89-4382-b201-7d0640395506/volumes"
Sep 19 11:47:22.595550 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:22.595531    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9" path="/var/lib/kubelet/pods/9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9/volumes"
Sep 19 11:47:22.595805 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:22.595792    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd21fff4-04ba-4e79-b7d6-200d906c5ea5" path="/var/lib/kubelet/pods/fd21fff4-04ba-4e79-b7d6-200d906c5ea5/volumes"
Sep 19 11:47:25.533960 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.533928    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"]
Sep 19 11:47:25.534381 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.534366    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd21fff4-04ba-4e79-b7d6-200d906c5ea5" containerName="revoke-keys"
Sep 19 11:47:25.534428 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.534383    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd21fff4-04ba-4e79-b7d6-200d906c5ea5" containerName="revoke-keys"
Sep 19 11:47:25.534428 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.534407    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ea9fc31-ab89-4382-b201-7d0640395506" containerName="payload-pre-processing"
Sep 19 11:47:25.534428 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.534414    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ea9fc31-ab89-4382-b201-7d0640395506" containerName="payload-pre-processing"
Sep 19 11:47:25.534428 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.534420    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9" containerName="payload-processing"
Sep 19 11:47:25.534428 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.534426    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9" containerName="payload-processing"
Sep 19 11:47:25.534582 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.534492    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ea9fc31-ab89-4382-b201-7d0640395506" containerName="payload-pre-processing"
Sep 19 11:47:25.534582 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.534505    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd21fff4-04ba-4e79-b7d6-200d906c5ea5" containerName="revoke-keys"
Sep 19 11:47:25.534582 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.534517    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9aa3c3e9-4096-49d0-8a5d-9c0a69ac78b9" containerName="payload-processing"
Sep 19 11:47:25.539014 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.538994    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"
Sep 19 11:47:25.541462 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.541429    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-a527d4-dockercfg-q8f2h\""
Sep 19 11:47:25.541558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.541430    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-a527d4\""
Sep 19 11:47:25.547601 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.547484    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"]
Sep 19 11:47:25.549691 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.549665    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85w2f\" (UniqueName: \"kubernetes.io/projected/39139ca0-df45-48ec-8bfb-856c2ad214be-kube-api-access-85w2f\") pod \"payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql\" (UID: \"39139ca0-df45-48ec-8bfb-856c2ad214be\") " pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"
Sep 19 11:47:25.549801 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.549749    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39139ca0-df45-48ec-8bfb-856c2ad214be-plugins-config-volume\") pod \"payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql\" (UID: \"39139ca0-df45-48ec-8bfb-856c2ad214be\") " pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"
Sep 19 11:47:25.650722 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.650691    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39139ca0-df45-48ec-8bfb-856c2ad214be-plugins-config-volume\") pod \"payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql\" (UID: \"39139ca0-df45-48ec-8bfb-856c2ad214be\") " pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"
Sep 19 11:47:25.650887 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.650837    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85w2f\" (UniqueName: \"kubernetes.io/projected/39139ca0-df45-48ec-8bfb-856c2ad214be-kube-api-access-85w2f\") pod \"payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql\" (UID: \"39139ca0-df45-48ec-8bfb-856c2ad214be\") " pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"
Sep 19 11:47:25.651457 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.651437    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39139ca0-df45-48ec-8bfb-856c2ad214be-plugins-config-volume\") pod \"payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql\" (UID: \"39139ca0-df45-48ec-8bfb-856c2ad214be\") " pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"
Sep 19 11:47:25.658255 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.658227    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85w2f\" (UniqueName: \"kubernetes.io/projected/39139ca0-df45-48ec-8bfb-856c2ad214be-kube-api-access-85w2f\") pod \"payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql\" (UID: \"39139ca0-df45-48ec-8bfb-856c2ad214be\") " pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"
Sep 19 11:47:25.851669 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:25.851594    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"
Sep 19 11:47:26.123807 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:26.123772    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"]
Sep 19 11:47:26.125148 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:47:26.125125    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39139ca0_df45_48ec_8bfb_856c2ad214be.slice/crio-d9b33105ecf840826b6929c7aa75c543efcdc9e560c2bbfed9c2f580d82b3c4b WatchSource:0}: Error finding container d9b33105ecf840826b6929c7aa75c543efcdc9e560c2bbfed9c2f580d82b3c4b: Status 404 returned error can't find the container with id d9b33105ecf840826b6929c7aa75c543efcdc9e560c2bbfed9c2f580d82b3c4b
Sep 19 11:47:26.908136 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:26.908099    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql" event={"ID":"39139ca0-df45-48ec-8bfb-856c2ad214be","Type":"ContainerStarted","Data":"b968106457d0e77b0769c7cdcf1e6dc5b695ee2741265fe0f5343a0aa1906fd4"}
Sep 19 11:47:26.908527 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:26.908163    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql" event={"ID":"39139ca0-df45-48ec-8bfb-856c2ad214be","Type":"ContainerStarted","Data":"d9b33105ecf840826b6929c7aa75c543efcdc9e560c2bbfed9c2f580d82b3c4b"}
Sep 19 11:47:26.908527 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:26.908225    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"
Sep 19 11:47:26.928391 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:26.928331    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql" podStartSLOduration=1.928311799 podStartE2EDuration="1.928311799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:47:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:47:26.925819497 +0000 UTC m=+798.848132378" watchObservedRunningTime="2026-09-19 11:47:26.928311799 +0000 UTC m=+798.850624682"
Sep 19 11:47:37.916891 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:47:37.916820    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"
Sep 19 11:48:43.293941 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:43.293889    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr"]
Sep 19 11:48:43.297775 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:43.297759    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr"
Sep 19 11:48:43.300102 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:43.300081    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:48:43.304910 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:43.304887    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr"]
Sep 19 11:48:43.376671 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:43.376644    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr\" (UID: \"4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr"
Sep 19 11:48:43.478118 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:43.478081    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr\" (UID: \"4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr"
Sep 19 11:48:43.478659 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:43.478639    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr\" (UID: \"4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr"
Sep 19 11:48:43.609393 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:43.609317    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr"
Sep 19 11:48:43.879644 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:43.879610    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr"]
Sep 19 11:48:43.881608 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:48:43.881579    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e9ff4dd_9c7a_4d95_9c77_02fb8f9c2a24.slice/crio-b7c08ad1478e31dad63846b4214fff5cb5a2f6ac4f1bbf894268b3aecce85951 WatchSource:0}: Error finding container b7c08ad1478e31dad63846b4214fff5cb5a2f6ac4f1bbf894268b3aecce85951: Status 404 returned error can't find the container with id b7c08ad1478e31dad63846b4214fff5cb5a2f6ac4f1bbf894268b3aecce85951
Sep 19 11:48:44.347936 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:44.347901    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr" event={"ID":"4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24","Type":"ContainerStarted","Data":"bdf28b97560c58e5045d38cfcd73d30b40afcf160537407876b5530336948138"}
Sep 19 11:48:44.348320 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:44.347951    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr" event={"ID":"4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24","Type":"ContainerStarted","Data":"b7c08ad1478e31dad63846b4214fff5cb5a2f6ac4f1bbf894268b3aecce85951"}
Sep 19 11:48:44.362499 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:44.362435    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr" podStartSLOduration=1.362419176 podStartE2EDuration="1.362419176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:48:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:48:44.360592604 +0000 UTC m=+876.282905488" watchObservedRunningTime="2026-09-19 11:48:44.362419176 +0000 UTC m=+876.284732057"
Sep 19 11:48:45.355800 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:45.355767    2559 generic.go:356] "Generic (PLEG): container finished" podID="4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24" containerID="bdf28b97560c58e5045d38cfcd73d30b40afcf160537407876b5530336948138" exitCode=0
Sep 19 11:48:45.356216 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:45.355817    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr" event={"ID":"4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24","Type":"ContainerDied","Data":"bdf28b97560c58e5045d38cfcd73d30b40afcf160537407876b5530336948138"}
Sep 19 11:48:46.501866 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:46.501842    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr"
Sep 19 11:48:46.606628 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:46.606597    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24-maas-api-service-ca\") pod \"4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24\" (UID: \"4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24\") "
Sep 19 11:48:46.606927 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:46.606905    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24" (UID: "4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:48:46.708125 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:46.708095    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:48:47.369251 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:47.369225    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr"
Sep 19 11:48:47.369251 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:47.369242    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr" event={"ID":"4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24","Type":"ContainerDied","Data":"b7c08ad1478e31dad63846b4214fff5cb5a2f6ac4f1bbf894268b3aecce85951"}
Sep 19 11:48:47.369522 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:47.369284    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7c08ad1478e31dad63846b4214fff5cb5a2f6ac4f1bbf894268b3aecce85951"
Sep 19 11:48:53.988781 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:53.988741    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr"]
Sep 19 11:48:53.990886 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:53.990865    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c4de155d-d8dxr"]
Sep 19 11:48:54.596471 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:48:54.596433    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24" path="/var/lib/kubelet/pods/4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24/volumes"
Sep 19 11:49:12.474093 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:12.474061    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc"]
Sep 19 11:49:12.476589 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:12.474697    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24" containerName="revoke-keys"
Sep 19 11:49:12.476589 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:12.474715    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24" containerName="revoke-keys"
Sep 19 11:49:12.476589 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:12.474810    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e9ff4dd-9c7a-4d95-9c77-02fb8f9c2a24" containerName="revoke-keys"
Sep 19 11:49:12.477809 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:12.477794    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc"
Sep 19 11:49:12.480164 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:12.480143    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:49:12.489904 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:12.489874    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc"]
Sep 19 11:49:12.545121 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:12.545090    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d362838-d287-4966-b560-738c15318ab9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-a527d4-7swwc\" (UID: \"4d362838-d287-4966-b560-738c15318ab9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc"
Sep 19 11:49:12.646827 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:12.646789    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d362838-d287-4966-b560-738c15318ab9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-a527d4-7swwc\" (UID: \"4d362838-d287-4966-b560-738c15318ab9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc"
Sep 19 11:49:12.647543 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:12.647521    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d362838-d287-4966-b560-738c15318ab9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-a527d4-7swwc\" (UID: \"4d362838-d287-4966-b560-738c15318ab9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc"
Sep 19 11:49:12.790988 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:12.790915    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc"
Sep 19 11:49:13.071970 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:13.070437    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc"]
Sep 19 11:49:13.078044 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:49:13.078005    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d362838_d287_4966_b560_738c15318ab9.slice/crio-3f6215227573c5b12c9dae50462fec8986df5922d6a148780ab84689a459ff89 WatchSource:0}: Error finding container 3f6215227573c5b12c9dae50462fec8986df5922d6a148780ab84689a459ff89: Status 404 returned error can't find the container with id 3f6215227573c5b12c9dae50462fec8986df5922d6a148780ab84689a459ff89
Sep 19 11:49:13.525111 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:13.525071    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc" event={"ID":"4d362838-d287-4966-b560-738c15318ab9","Type":"ContainerStarted","Data":"cc964284e45d6d6bd7d95e7b78d44ed2382a80b378e3cb872ebfe2cf89cd03a2"}
Sep 19 11:49:13.525514 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:13.525131    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc" event={"ID":"4d362838-d287-4966-b560-738c15318ab9","Type":"ContainerStarted","Data":"3f6215227573c5b12c9dae50462fec8986df5922d6a148780ab84689a459ff89"}
Sep 19 11:49:13.541454 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:13.541404    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc" podStartSLOduration=1.5413859049999998 podStartE2EDuration="1.541385905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:49:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:49:13.538762666 +0000 UTC m=+905.461075546" watchObservedRunningTime="2026-09-19 11:49:13.541385905 +0000 UTC m=+905.463698785"
Sep 19 11:49:14.532696 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:14.532661    2559 generic.go:356] "Generic (PLEG): container finished" podID="4d362838-d287-4966-b560-738c15318ab9" containerID="cc964284e45d6d6bd7d95e7b78d44ed2382a80b378e3cb872ebfe2cf89cd03a2" exitCode=0
Sep 19 11:49:14.533092 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:14.532697    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc" event={"ID":"4d362838-d287-4966-b560-738c15318ab9","Type":"ContainerDied","Data":"cc964284e45d6d6bd7d95e7b78d44ed2382a80b378e3cb872ebfe2cf89cd03a2"}
Sep 19 11:49:15.675536 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:15.675513    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc"
Sep 19 11:49:15.777649 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:15.777621    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d362838-d287-4966-b560-738c15318ab9-maas-api-service-ca\") pod \"4d362838-d287-4966-b560-738c15318ab9\" (UID: \"4d362838-d287-4966-b560-738c15318ab9\") "
Sep 19 11:49:15.777966 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:15.777942    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d362838-d287-4966-b560-738c15318ab9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d362838-d287-4966-b560-738c15318ab9" (UID: "4d362838-d287-4966-b560-738c15318ab9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:49:15.879092 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:15.879021    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d362838-d287-4966-b560-738c15318ab9-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:49:16.545590 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:16.545562    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc"
Sep 19 11:49:16.545590 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:16.545571    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc" event={"ID":"4d362838-d287-4966-b560-738c15318ab9","Type":"ContainerDied","Data":"3f6215227573c5b12c9dae50462fec8986df5922d6a148780ab84689a459ff89"}
Sep 19 11:49:16.545798 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:16.545643    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f6215227573c5b12c9dae50462fec8986df5922d6a148780ab84689a459ff89"
Sep 19 11:49:22.171727 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.171691    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"]
Sep 19 11:49:22.172172 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.172158    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d362838-d287-4966-b560-738c15318ab9" containerName="revoke-keys"
Sep 19 11:49:22.172251 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.172175    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d362838-d287-4966-b560-738c15318ab9" containerName="revoke-keys"
Sep 19 11:49:22.172300 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.172279    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d362838-d287-4966-b560-738c15318ab9" containerName="revoke-keys"
Sep 19 11:49:22.175507 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.175491    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"
Sep 19 11:49:22.177977 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.177954    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-4bb32d-dockercfg-b7whl\""
Sep 19 11:49:22.178164 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.178148    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-4bb32d\""
Sep 19 11:49:22.185934 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.185912    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"]
Sep 19 11:49:22.343398 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.343366    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7a3758f-2148-48fc-abf8-a6b9ef581726-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84\" (UID: \"f7a3758f-2148-48fc-abf8-a6b9ef581726\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"
Sep 19 11:49:22.343574 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.343416    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79kss\" (UniqueName: \"kubernetes.io/projected/f7a3758f-2148-48fc-abf8-a6b9ef581726-kube-api-access-79kss\") pod \"payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84\" (UID: \"f7a3758f-2148-48fc-abf8-a6b9ef581726\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"
Sep 19 11:49:22.444826 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.444797    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7a3758f-2148-48fc-abf8-a6b9ef581726-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84\" (UID: \"f7a3758f-2148-48fc-abf8-a6b9ef581726\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"
Sep 19 11:49:22.444977 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.444849    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79kss\" (UniqueName: \"kubernetes.io/projected/f7a3758f-2148-48fc-abf8-a6b9ef581726-kube-api-access-79kss\") pod \"payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84\" (UID: \"f7a3758f-2148-48fc-abf8-a6b9ef581726\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"
Sep 19 11:49:22.445565 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.445545    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7a3758f-2148-48fc-abf8-a6b9ef581726-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84\" (UID: \"f7a3758f-2148-48fc-abf8-a6b9ef581726\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"
Sep 19 11:49:22.453353 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.453329    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79kss\" (UniqueName: \"kubernetes.io/projected/f7a3758f-2148-48fc-abf8-a6b9ef581726-kube-api-access-79kss\") pod \"payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84\" (UID: \"f7a3758f-2148-48fc-abf8-a6b9ef581726\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"
Sep 19 11:49:22.488017 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.487996    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"
Sep 19 11:49:22.762548 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.761342    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"]
Sep 19 11:49:22.936320 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.936282    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"]
Sep 19 11:49:22.936586 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:22.936557    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql" podUID="39139ca0-df45-48ec-8bfb-856c2ad214be" containerName="payload-processing" containerID="cri-o://b968106457d0e77b0769c7cdcf1e6dc5b695ee2741265fe0f5343a0aa1906fd4" gracePeriod=30
Sep 19 11:49:23.097158 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.097117    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc"]
Sep 19 11:49:23.100435 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.100405    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a527d4-7swwc"]
Sep 19 11:49:23.188960 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.188931    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"
Sep 19 11:49:23.353654 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.353580    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39139ca0-df45-48ec-8bfb-856c2ad214be-plugins-config-volume\") pod \"39139ca0-df45-48ec-8bfb-856c2ad214be\" (UID: \"39139ca0-df45-48ec-8bfb-856c2ad214be\") "
Sep 19 11:49:23.353793 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.353678    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85w2f\" (UniqueName: \"kubernetes.io/projected/39139ca0-df45-48ec-8bfb-856c2ad214be-kube-api-access-85w2f\") pod \"39139ca0-df45-48ec-8bfb-856c2ad214be\" (UID: \"39139ca0-df45-48ec-8bfb-856c2ad214be\") "
Sep 19 11:49:23.353926 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.353901    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39139ca0-df45-48ec-8bfb-856c2ad214be-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "39139ca0-df45-48ec-8bfb-856c2ad214be" (UID: "39139ca0-df45-48ec-8bfb-856c2ad214be"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:49:23.355741 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.355722    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39139ca0-df45-48ec-8bfb-856c2ad214be-kube-api-access-85w2f" (OuterVolumeSpecName: "kube-api-access-85w2f") pod "39139ca0-df45-48ec-8bfb-856c2ad214be" (UID: "39139ca0-df45-48ec-8bfb-856c2ad214be"). InnerVolumeSpecName "kube-api-access-85w2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:49:23.455432 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.455406    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39139ca0-df45-48ec-8bfb-856c2ad214be-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:49:23.455432 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.455430    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85w2f\" (UniqueName: \"kubernetes.io/projected/39139ca0-df45-48ec-8bfb-856c2ad214be-kube-api-access-85w2f\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:49:23.589547 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.589506    2559 generic.go:356] "Generic (PLEG): container finished" podID="39139ca0-df45-48ec-8bfb-856c2ad214be" containerID="b968106457d0e77b0769c7cdcf1e6dc5b695ee2741265fe0f5343a0aa1906fd4" exitCode=0
Sep 19 11:49:23.589710 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.589557    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"
Sep 19 11:49:23.589710 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.589587    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql" event={"ID":"39139ca0-df45-48ec-8bfb-856c2ad214be","Type":"ContainerDied","Data":"b968106457d0e77b0769c7cdcf1e6dc5b695ee2741265fe0f5343a0aa1906fd4"}
Sep 19 11:49:23.589710 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.589634    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql" event={"ID":"39139ca0-df45-48ec-8bfb-856c2ad214be","Type":"ContainerDied","Data":"d9b33105ecf840826b6929c7aa75c543efcdc9e560c2bbfed9c2f580d82b3c4b"}
Sep 19 11:49:23.589710 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.589655    2559 scope.go:117] "RemoveContainer" containerID="b968106457d0e77b0769c7cdcf1e6dc5b695ee2741265fe0f5343a0aa1906fd4"
Sep 19 11:49:23.591567 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.591529    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84" event={"ID":"f7a3758f-2148-48fc-abf8-a6b9ef581726","Type":"ContainerStarted","Data":"fbcc9cffcb1473852194222d44a51e08d62414496b27f7c57e73f064982ba6ce"}
Sep 19 11:49:23.591708 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.591586    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84" event={"ID":"f7a3758f-2148-48fc-abf8-a6b9ef581726","Type":"ContainerStarted","Data":"9e95deddb89f1a42f27b6c9a8d4adddc537468153d78acc2c14f666dde096e63"}
Sep 19 11:49:23.591708 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.591651    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"
Sep 19 11:49:23.602226 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.602182    2559 scope.go:117] "RemoveContainer" containerID="b968106457d0e77b0769c7cdcf1e6dc5b695ee2741265fe0f5343a0aa1906fd4"
Sep 19 11:49:23.602601 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:49:23.602571    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b968106457d0e77b0769c7cdcf1e6dc5b695ee2741265fe0f5343a0aa1906fd4\": container with ID starting with b968106457d0e77b0769c7cdcf1e6dc5b695ee2741265fe0f5343a0aa1906fd4 not found: ID does not exist" containerID="b968106457d0e77b0769c7cdcf1e6dc5b695ee2741265fe0f5343a0aa1906fd4"
Sep 19 11:49:23.602707 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.602606    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b968106457d0e77b0769c7cdcf1e6dc5b695ee2741265fe0f5343a0aa1906fd4"} err="failed to get container status \"b968106457d0e77b0769c7cdcf1e6dc5b695ee2741265fe0f5343a0aa1906fd4\": rpc error: code = NotFound desc = could not find container \"b968106457d0e77b0769c7cdcf1e6dc5b695ee2741265fe0f5343a0aa1906fd4\": container with ID starting with b968106457d0e77b0769c7cdcf1e6dc5b695ee2741265fe0f5343a0aa1906fd4 not found: ID does not exist"
Sep 19 11:49:23.608345 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.608267    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84" podStartSLOduration=1.608250342 podStartE2EDuration="1.608250342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:49:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:49:23.607858431 +0000 UTC m=+915.530171315" watchObservedRunningTime="2026-09-19 11:49:23.608250342 +0000 UTC m=+915.530563222"
Sep 19 11:49:23.624172 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.624151    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"]
Sep 19 11:49:23.627350 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:23.627330    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a527d4-6d6d98f449-7hpql"]
Sep 19 11:49:24.596559 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:24.596519    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39139ca0-df45-48ec-8bfb-856c2ad214be" path="/var/lib/kubelet/pods/39139ca0-df45-48ec-8bfb-856c2ad214be/volumes"
Sep 19 11:49:24.597135 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:24.597008    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d362838-d287-4966-b560-738c15318ab9" path="/var/lib/kubelet/pods/4d362838-d287-4966-b560-738c15318ab9/volumes"
Sep 19 11:49:30.025904 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.025870    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"]
Sep 19 11:49:30.026341 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.026328    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39139ca0-df45-48ec-8bfb-856c2ad214be" containerName="payload-processing"
Sep 19 11:49:30.026341 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.026342    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="39139ca0-df45-48ec-8bfb-856c2ad214be" containerName="payload-processing"
Sep 19 11:49:30.026435 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.026413    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="39139ca0-df45-48ec-8bfb-856c2ad214be" containerName="payload-processing"
Sep 19 11:49:30.030957 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.030931    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.033430 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.033407    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-6e19b2-openshift-default-dockercfg-p748g\""
Sep 19 11:49:30.041899 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.041860    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"]
Sep 19 11:49:30.225212 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.225154    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-istio-envoy\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.225212 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.225214    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2qvj\" (UniqueName: \"kubernetes.io/projected/17fa9957-6197-461e-81a0-e6b305e549c9-kube-api-access-v2qvj\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.225420 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.225312    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17fa9957-6197-461e-81a0-e6b305e549c9-istio-token\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.225420 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.225369    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-workload-socket\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.225420 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.225395    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17fa9957-6197-461e-81a0-e6b305e549c9-istio-podinfo\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.225535 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.225421    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-workload-certs\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.225535 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.225452    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-credential-socket\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.225535 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.225468    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-istio-data\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.225535 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.225518    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17fa9957-6197-461e-81a0-e6b305e549c9-istiod-ca-cert\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.326485 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.326410    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-istio-envoy\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.326485 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.326444    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2qvj\" (UniqueName: \"kubernetes.io/projected/17fa9957-6197-461e-81a0-e6b305e549c9-kube-api-access-v2qvj\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.326485 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.326481    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17fa9957-6197-461e-81a0-e6b305e549c9-istio-token\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.326771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.326517    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-workload-socket\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.326771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.326538    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17fa9957-6197-461e-81a0-e6b305e549c9-istio-podinfo\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.326771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.326563    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-workload-certs\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.326771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.326668    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-credential-socket\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.326771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.326710    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-istio-data\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.326771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.326753    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17fa9957-6197-461e-81a0-e6b305e549c9-istiod-ca-cert\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.327115 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.326963    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-workload-socket\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.327115 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.327001    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-workload-certs\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.327115 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.327036    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-credential-socket\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.327284 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.327163    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-istio-data\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.327450 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.327434    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17fa9957-6197-461e-81a0-e6b305e549c9-istiod-ca-cert\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.329403 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.329380    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-istio-envoy\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.329608 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.329593    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17fa9957-6197-461e-81a0-e6b305e549c9-istio-podinfo\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.334159 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.334134    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17fa9957-6197-461e-81a0-e6b305e549c9-istio-token\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.334494 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.334476    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2qvj\" (UniqueName: \"kubernetes.io/projected/17fa9957-6197-461e-81a0-e6b305e549c9-kube-api-access-v2qvj\") pod \"e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") " pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.344411 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.344388    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:30.621726 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.621702    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"]
Sep 19 11:49:30.623817 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:49:30.623792    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17fa9957_6197_461e_81a0_e6b305e549c9.slice/crio-c6ee15fdb6c133de53f28b71972a6666d246012211d988e2a62b97d6824fb180 WatchSource:0}: Error finding container c6ee15fdb6c133de53f28b71972a6666d246012211d988e2a62b97d6824fb180: Status 404 returned error can't find the container with id c6ee15fdb6c133de53f28b71972a6666d246012211d988e2a62b97d6824fb180
Sep 19 11:49:30.626503 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.626470    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:49:30.626631 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.626535    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:49:30.626631 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.626571    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:49:30.633781 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:30.633760    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr" event={"ID":"17fa9957-6197-461e-81a0-e6b305e549c9","Type":"ContainerStarted","Data":"c6ee15fdb6c133de53f28b71972a6666d246012211d988e2a62b97d6824fb180"}
Sep 19 11:49:31.641252 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:31.641221    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr" event={"ID":"17fa9957-6197-461e-81a0-e6b305e549c9","Type":"ContainerStarted","Data":"f7dff116542e943a60d2eca3094622f0ef67c0ba5f251dc7477c8d103b8834ab"}
Sep 19 11:49:31.667218 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:31.662673    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr" podStartSLOduration=1.662653218 podStartE2EDuration="1.662653218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:49:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:49:31.659777325 +0000 UTC m=+923.582090207" watchObservedRunningTime="2026-09-19 11:49:31.662653218 +0000 UTC m=+923.584966101"
Sep 19 11:49:32.345265 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:32.345229    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:32.350649 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:32.350626    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:32.647771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:32.647688    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:32.648617 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:32.648594    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:49:34.600928 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:34.600905    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"
Sep 19 11:49:55.996146 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:55.996109    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"]
Sep 19 11:49:55.999784 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:55.999769    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"
Sep 19 11:49:56.002036 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.002013    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-176c86\""
Sep 19 11:49:56.002163 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.002040    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-176c86-dockercfg-g28hl\""
Sep 19 11:49:56.009808 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.009787    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"]
Sep 19 11:49:56.029145 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.029122    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"]
Sep 19 11:49:56.032841 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.032826    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"
Sep 19 11:49:56.041723 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.041698    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"]
Sep 19 11:49:56.069815 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.069787    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kb8j\" (UniqueName: \"kubernetes.io/projected/d8a66d1e-bd04-44e2-b5b5-8357b6734a47-kube-api-access-8kb8j\") pod \"payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb\" (UID: \"d8a66d1e-bd04-44e2-b5b5-8357b6734a47\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"
Sep 19 11:49:56.069916 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.069890    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8a66d1e-bd04-44e2-b5b5-8357b6734a47-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb\" (UID: \"d8a66d1e-bd04-44e2-b5b5-8357b6734a47\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"
Sep 19 11:49:56.170907 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.170876    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkkcz\" (UniqueName: \"kubernetes.io/projected/425ddf7d-30d9-46fe-b0c9-0516bf013f8c-kube-api-access-rkkcz\") pod \"payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f\" (UID: \"425ddf7d-30d9-46fe-b0c9-0516bf013f8c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"
Sep 19 11:49:56.171064 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.170959    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/425ddf7d-30d9-46fe-b0c9-0516bf013f8c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f\" (UID: \"425ddf7d-30d9-46fe-b0c9-0516bf013f8c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"
Sep 19 11:49:56.171064 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.170986    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kb8j\" (UniqueName: \"kubernetes.io/projected/d8a66d1e-bd04-44e2-b5b5-8357b6734a47-kube-api-access-8kb8j\") pod \"payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb\" (UID: \"d8a66d1e-bd04-44e2-b5b5-8357b6734a47\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"
Sep 19 11:49:56.171143 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.171061    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8a66d1e-bd04-44e2-b5b5-8357b6734a47-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb\" (UID: \"d8a66d1e-bd04-44e2-b5b5-8357b6734a47\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"
Sep 19 11:49:56.171701 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.171683    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8a66d1e-bd04-44e2-b5b5-8357b6734a47-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb\" (UID: \"d8a66d1e-bd04-44e2-b5b5-8357b6734a47\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"
Sep 19 11:49:56.178367 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.178338    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kb8j\" (UniqueName: \"kubernetes.io/projected/d8a66d1e-bd04-44e2-b5b5-8357b6734a47-kube-api-access-8kb8j\") pod \"payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb\" (UID: \"d8a66d1e-bd04-44e2-b5b5-8357b6734a47\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"
Sep 19 11:49:56.271655 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.271595    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/425ddf7d-30d9-46fe-b0c9-0516bf013f8c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f\" (UID: \"425ddf7d-30d9-46fe-b0c9-0516bf013f8c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"
Sep 19 11:49:56.271796 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.271664    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkkcz\" (UniqueName: \"kubernetes.io/projected/425ddf7d-30d9-46fe-b0c9-0516bf013f8c-kube-api-access-rkkcz\") pod \"payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f\" (UID: \"425ddf7d-30d9-46fe-b0c9-0516bf013f8c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"
Sep 19 11:49:56.272326 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.272305    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/425ddf7d-30d9-46fe-b0c9-0516bf013f8c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f\" (UID: \"425ddf7d-30d9-46fe-b0c9-0516bf013f8c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"
Sep 19 11:49:56.279555 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.279532    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkkcz\" (UniqueName: \"kubernetes.io/projected/425ddf7d-30d9-46fe-b0c9-0516bf013f8c-kube-api-access-rkkcz\") pod \"payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f\" (UID: \"425ddf7d-30d9-46fe-b0c9-0516bf013f8c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"
Sep 19 11:49:56.312332 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.312307    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"
Sep 19 11:49:56.344624 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.344595    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"
Sep 19 11:49:56.585402 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.585372    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"]
Sep 19 11:49:56.585525 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:49:56.585422    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8a66d1e_bd04_44e2_b5b5_8357b6734a47.slice/crio-347ec797c949c7a495c8cf247efd0f0912c97b1f333bf0334b097fb9cc35b28e WatchSource:0}: Error finding container 347ec797c949c7a495c8cf247efd0f0912c97b1f333bf0334b097fb9cc35b28e: Status 404 returned error can't find the container with id 347ec797c949c7a495c8cf247efd0f0912c97b1f333bf0334b097fb9cc35b28e
Sep 19 11:49:56.630359 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.630335    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"]
Sep 19 11:49:56.633316 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:49:56.633292    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod425ddf7d_30d9_46fe_b0c9_0516bf013f8c.slice/crio-dfd277a312cedbe57a61b0ed24632135fe25fe4aea955d83ff834644e7384de7 WatchSource:0}: Error finding container dfd277a312cedbe57a61b0ed24632135fe25fe4aea955d83ff834644e7384de7: Status 404 returned error can't find the container with id dfd277a312cedbe57a61b0ed24632135fe25fe4aea955d83ff834644e7384de7
Sep 19 11:49:56.788652 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.788616    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb" event={"ID":"d8a66d1e-bd04-44e2-b5b5-8357b6734a47","Type":"ContainerStarted","Data":"fb72f1f45d447bdc9fd1741594f7aab1d944557d445a8f510872c99858e50909"}
Sep 19 11:49:56.788840 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.788670    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb" event={"ID":"d8a66d1e-bd04-44e2-b5b5-8357b6734a47","Type":"ContainerStarted","Data":"347ec797c949c7a495c8cf247efd0f0912c97b1f333bf0334b097fb9cc35b28e"}
Sep 19 11:49:56.788903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.788842    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"
Sep 19 11:49:56.790343 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.790319    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f" event={"ID":"425ddf7d-30d9-46fe-b0c9-0516bf013f8c","Type":"ContainerStarted","Data":"a248aecf2107ced7d6433d96bc1c16a2f8c95722c1feab187b0a44d105af20ec"}
Sep 19 11:49:56.790467 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.790360    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f" event={"ID":"425ddf7d-30d9-46fe-b0c9-0516bf013f8c","Type":"ContainerStarted","Data":"dfd277a312cedbe57a61b0ed24632135fe25fe4aea955d83ff834644e7384de7"}
Sep 19 11:49:56.790529 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.790472    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"
Sep 19 11:49:56.805134 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.805089    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb" podStartSLOduration=1.805074681 podStartE2EDuration="1.805074681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:49:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:49:56.803249787 +0000 UTC m=+948.725562668" watchObservedRunningTime="2026-09-19 11:49:56.805074681 +0000 UTC m=+948.727387585"
Sep 19 11:49:56.819091 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:49:56.819047    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f" podStartSLOduration=0.819035878 podStartE2EDuration="819.035878ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:49:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:49:56.818252409 +0000 UTC m=+948.740565289" watchObservedRunningTime="2026-09-19 11:49:56.819035878 +0000 UTC m=+948.741348759"
Sep 19 11:50:07.799895 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:07.799866    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"
Sep 19 11:50:07.800307 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:07.799979    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"
Sep 19 11:50:39.417538 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:39.416478    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"]
Sep 19 11:50:39.421810 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:39.421785    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"
Sep 19 11:50:39.423901 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:39.423877    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"]
Sep 19 11:50:39.424027 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:39.423961    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-fcfc8fcc-dockercfg-9s669\""
Sep 19 11:50:39.424090 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:39.424072    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-fcfc8fcc\""
Sep 19 11:50:39.485581 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:39.485532    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d01315b-0855-433a-8e81-337384350872-plugins-config-volume\") pod \"payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5\" (UID: \"1d01315b-0855-433a-8e81-337384350872\") " pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"
Sep 19 11:50:39.485766 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:39.485635    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b7x6\" (UniqueName: \"kubernetes.io/projected/1d01315b-0855-433a-8e81-337384350872-kube-api-access-8b7x6\") pod \"payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5\" (UID: \"1d01315b-0855-433a-8e81-337384350872\") " pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"
Sep 19 11:50:39.586874 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:39.586842    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d01315b-0855-433a-8e81-337384350872-plugins-config-volume\") pod \"payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5\" (UID: \"1d01315b-0855-433a-8e81-337384350872\") " pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"
Sep 19 11:50:39.587045 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:39.586908    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b7x6\" (UniqueName: \"kubernetes.io/projected/1d01315b-0855-433a-8e81-337384350872-kube-api-access-8b7x6\") pod \"payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5\" (UID: \"1d01315b-0855-433a-8e81-337384350872\") " pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"
Sep 19 11:50:39.587568 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:39.587546    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d01315b-0855-433a-8e81-337384350872-plugins-config-volume\") pod \"payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5\" (UID: \"1d01315b-0855-433a-8e81-337384350872\") " pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"
Sep 19 11:50:39.594555 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:39.594523    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b7x6\" (UniqueName: \"kubernetes.io/projected/1d01315b-0855-433a-8e81-337384350872-kube-api-access-8b7x6\") pod \"payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5\" (UID: \"1d01315b-0855-433a-8e81-337384350872\") " pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"
Sep 19 11:50:39.734030 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:39.734005    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"
Sep 19 11:50:40.030238 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:40.030211    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"]
Sep 19 11:50:40.032006 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:50:40.031979    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d01315b_0855_433a_8e81_337384350872.slice/crio-17a15db32077b8edc3001ab628dc3d6d7b41cff8c4b8538ff86295f90eb4b05d WatchSource:0}: Error finding container 17a15db32077b8edc3001ab628dc3d6d7b41cff8c4b8538ff86295f90eb4b05d: Status 404 returned error can't find the container with id 17a15db32077b8edc3001ab628dc3d6d7b41cff8c4b8538ff86295f90eb4b05d
Sep 19 11:50:40.058813 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:40.058785    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5" event={"ID":"1d01315b-0855-433a-8e81-337384350872","Type":"ContainerStarted","Data":"17a15db32077b8edc3001ab628dc3d6d7b41cff8c4b8538ff86295f90eb4b05d"}
Sep 19 11:50:41.066198 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:41.066156    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5" event={"ID":"1d01315b-0855-433a-8e81-337384350872","Type":"ContainerStarted","Data":"6f3f37d79d3b3cd8548d398653553da10af93eb880c263b552ef88f14a3e463b"}
Sep 19 11:50:41.066587 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:41.066254    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"
Sep 19 11:50:41.081628 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:41.081574    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5" podStartSLOduration=2.081558917 podStartE2EDuration="2.081558917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:50:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:50:41.080515503 +0000 UTC m=+993.002828386" watchObservedRunningTime="2026-09-19 11:50:41.081558917 +0000 UTC m=+993.003871799"
Sep 19 11:50:52.074844 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:52.074811    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"
Sep 19 11:50:54.377100 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:54.377062    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"]
Sep 19 11:50:54.377591 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:54.377326    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr" podUID="17fa9957-6197-461e-81a0-e6b305e549c9" containerName="istio-proxy" containerID="cri-o://f7dff116542e943a60d2eca3094622f0ef67c0ba5f251dc7477c8d103b8834ab" gracePeriod=30
Sep 19 11:50:55.422118 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:55.422086    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s"]
Sep 19 11:50:55.425467 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:55.425451    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s"
Sep 19 11:50:55.428000 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:55.427977    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:50:55.432816 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:55.432790    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s"]
Sep 19 11:50:55.540955 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:55.540924    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b28c414b-65da-4e87-96d4-95afb3357103-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s\" (UID: \"b28c414b-65da-4e87-96d4-95afb3357103\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s"
Sep 19 11:50:55.642709 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:55.642676    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b28c414b-65da-4e87-96d4-95afb3357103-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s\" (UID: \"b28c414b-65da-4e87-96d4-95afb3357103\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s"
Sep 19 11:50:55.643771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:55.643743    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b28c414b-65da-4e87-96d4-95afb3357103-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s\" (UID: \"b28c414b-65da-4e87-96d4-95afb3357103\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s"
Sep 19 11:50:55.737881 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:55.737850    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s"
Sep 19 11:50:56.014468 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:56.014442    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s"]
Sep 19 11:50:56.015896 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:50:56.015865    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb28c414b_65da_4e87_96d4_95afb3357103.slice/crio-01dd919b47a5ef6572ac890a5af6aed2eea74bfc5efd3a771e11ea62436cefb0 WatchSource:0}: Error finding container 01dd919b47a5ef6572ac890a5af6aed2eea74bfc5efd3a771e11ea62436cefb0: Status 404 returned error can't find the container with id 01dd919b47a5ef6572ac890a5af6aed2eea74bfc5efd3a771e11ea62436cefb0
Sep 19 11:50:56.158838 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:56.158805    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s" event={"ID":"b28c414b-65da-4e87-96d4-95afb3357103","Type":"ContainerStarted","Data":"01dd919b47a5ef6572ac890a5af6aed2eea74bfc5efd3a771e11ea62436cefb0"}
Sep 19 11:50:57.166823 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:57.166734    2559 generic.go:356] "Generic (PLEG): container finished" podID="b28c414b-65da-4e87-96d4-95afb3357103" containerID="194a4ae258553fccef4476438634b62f37553021452201b0dce85c79b9155485" exitCode=0
Sep 19 11:50:57.167172 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:57.166821    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s" event={"ID":"b28c414b-65da-4e87-96d4-95afb3357103","Type":"ContainerDied","Data":"194a4ae258553fccef4476438634b62f37553021452201b0dce85c79b9155485"}
Sep 19 11:50:58.315528 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:58.315503    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s"
Sep 19 11:50:58.370493 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:58.370459    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b28c414b-65da-4e87-96d4-95afb3357103-maas-api-service-ca\") pod \"b28c414b-65da-4e87-96d4-95afb3357103\" (UID: \"b28c414b-65da-4e87-96d4-95afb3357103\") "
Sep 19 11:50:58.370902 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:58.370876    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b28c414b-65da-4e87-96d4-95afb3357103-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b28c414b-65da-4e87-96d4-95afb3357103" (UID: "b28c414b-65da-4e87-96d4-95afb3357103"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:50:58.471358 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:58.471330    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b28c414b-65da-4e87-96d4-95afb3357103-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:50:59.180271 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.180235    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s"
Sep 19 11:50:59.180271 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.180249    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s" event={"ID":"b28c414b-65da-4e87-96d4-95afb3357103","Type":"ContainerDied","Data":"01dd919b47a5ef6572ac890a5af6aed2eea74bfc5efd3a771e11ea62436cefb0"}
Sep 19 11:50:59.180489 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.180309    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01dd919b47a5ef6572ac890a5af6aed2eea74bfc5efd3a771e11ea62436cefb0"
Sep 19 11:50:59.663308 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.663019    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:50:59.783038 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.782939    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17fa9957-6197-461e-81a0-e6b305e549c9-istio-token\") pod \"17fa9957-6197-461e-81a0-e6b305e549c9\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") "
Sep 19 11:50:59.783038 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.782996    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2qvj\" (UniqueName: \"kubernetes.io/projected/17fa9957-6197-461e-81a0-e6b305e549c9-kube-api-access-v2qvj\") pod \"17fa9957-6197-461e-81a0-e6b305e549c9\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") "
Sep 19 11:50:59.783296 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.783121    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17fa9957-6197-461e-81a0-e6b305e549c9-istio-podinfo\") pod \"17fa9957-6197-461e-81a0-e6b305e549c9\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") "
Sep 19 11:50:59.783296 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.783157    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-workload-socket\") pod \"17fa9957-6197-461e-81a0-e6b305e549c9\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") "
Sep 19 11:50:59.783296 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.783228    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-credential-socket\") pod \"17fa9957-6197-461e-81a0-e6b305e549c9\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") "
Sep 19 11:50:59.783296 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.783272    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-workload-certs\") pod \"17fa9957-6197-461e-81a0-e6b305e549c9\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") "
Sep 19 11:50:59.783512 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.783307    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-istio-envoy\") pod \"17fa9957-6197-461e-81a0-e6b305e549c9\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") "
Sep 19 11:50:59.783512 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.783338    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17fa9957-6197-461e-81a0-e6b305e549c9-istiod-ca-cert\") pod \"17fa9957-6197-461e-81a0-e6b305e549c9\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") "
Sep 19 11:50:59.783512 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.783360    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "17fa9957-6197-461e-81a0-e6b305e549c9" (UID: "17fa9957-6197-461e-81a0-e6b305e549c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:50:59.783512 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.783385    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-istio-data\") pod \"17fa9957-6197-461e-81a0-e6b305e549c9\" (UID: \"17fa9957-6197-461e-81a0-e6b305e549c9\") "
Sep 19 11:50:59.783512 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.783456    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "17fa9957-6197-461e-81a0-e6b305e549c9" (UID: "17fa9957-6197-461e-81a0-e6b305e549c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:50:59.783764 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.783547    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-istio-data" (OuterVolumeSpecName: "istio-data") pod "17fa9957-6197-461e-81a0-e6b305e549c9" (UID: "17fa9957-6197-461e-81a0-e6b305e549c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:50:59.783826 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.783789    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-workload-socket\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:50:59.783826 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.783807    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-credential-socket\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:50:59.783826 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.783823    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-istio-data\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:50:59.784003 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.783980    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "17fa9957-6197-461e-81a0-e6b305e549c9" (UID: "17fa9957-6197-461e-81a0-e6b305e549c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:50:59.784003 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.783987    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17fa9957-6197-461e-81a0-e6b305e549c9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "17fa9957-6197-461e-81a0-e6b305e549c9" (UID: "17fa9957-6197-461e-81a0-e6b305e549c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:50:59.785554 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.785519    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17fa9957-6197-461e-81a0-e6b305e549c9-kube-api-access-v2qvj" (OuterVolumeSpecName: "kube-api-access-v2qvj") pod "17fa9957-6197-461e-81a0-e6b305e549c9" (UID: "17fa9957-6197-461e-81a0-e6b305e549c9"). InnerVolumeSpecName "kube-api-access-v2qvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:50:59.785848 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.785824    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17fa9957-6197-461e-81a0-e6b305e549c9-istio-token" (OuterVolumeSpecName: "istio-token") pod "17fa9957-6197-461e-81a0-e6b305e549c9" (UID: "17fa9957-6197-461e-81a0-e6b305e549c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:50:59.786080 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.786058    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "17fa9957-6197-461e-81a0-e6b305e549c9" (UID: "17fa9957-6197-461e-81a0-e6b305e549c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:50:59.786321 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.786299    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/17fa9957-6197-461e-81a0-e6b305e549c9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "17fa9957-6197-461e-81a0-e6b305e549c9" (UID: "17fa9957-6197-461e-81a0-e6b305e549c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 11:50:59.885078 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.885040    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17fa9957-6197-461e-81a0-e6b305e549c9-istio-podinfo\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:50:59.885078 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.885069    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-workload-certs\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:50:59.885078 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.885079    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17fa9957-6197-461e-81a0-e6b305e549c9-istio-envoy\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:50:59.885078 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.885087    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17fa9957-6197-461e-81a0-e6b305e549c9-istiod-ca-cert\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:50:59.885357 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.885094    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17fa9957-6197-461e-81a0-e6b305e549c9-istio-token\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:50:59.885357 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:50:59.885102    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2qvj\" (UniqueName: \"kubernetes.io/projected/17fa9957-6197-461e-81a0-e6b305e549c9-kube-api-access-v2qvj\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:51:00.188165 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:00.188133    2559 generic.go:356] "Generic (PLEG): container finished" podID="17fa9957-6197-461e-81a0-e6b305e549c9" containerID="f7dff116542e943a60d2eca3094622f0ef67c0ba5f251dc7477c8d103b8834ab" exitCode=0
Sep 19 11:51:00.188338 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:00.188173    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"
Sep 19 11:51:00.188338 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:00.188177    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr" event={"ID":"17fa9957-6197-461e-81a0-e6b305e549c9","Type":"ContainerDied","Data":"f7dff116542e943a60d2eca3094622f0ef67c0ba5f251dc7477c8d103b8834ab"}
Sep 19 11:51:00.188338 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:00.188254    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr" event={"ID":"17fa9957-6197-461e-81a0-e6b305e549c9","Type":"ContainerDied","Data":"c6ee15fdb6c133de53f28b71972a6666d246012211d988e2a62b97d6824fb180"}
Sep 19 11:51:00.188338 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:00.188276    2559 scope.go:117] "RemoveContainer" containerID="f7dff116542e943a60d2eca3094622f0ef67c0ba5f251dc7477c8d103b8834ab"
Sep 19 11:51:00.198948 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:00.198934    2559 scope.go:117] "RemoveContainer" containerID="f7dff116542e943a60d2eca3094622f0ef67c0ba5f251dc7477c8d103b8834ab"
Sep 19 11:51:00.199306 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:51:00.199286    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7dff116542e943a60d2eca3094622f0ef67c0ba5f251dc7477c8d103b8834ab\": container with ID starting with f7dff116542e943a60d2eca3094622f0ef67c0ba5f251dc7477c8d103b8834ab not found: ID does not exist" containerID="f7dff116542e943a60d2eca3094622f0ef67c0ba5f251dc7477c8d103b8834ab"
Sep 19 11:51:00.199347 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:00.199314    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7dff116542e943a60d2eca3094622f0ef67c0ba5f251dc7477c8d103b8834ab"} err="failed to get container status \"f7dff116542e943a60d2eca3094622f0ef67c0ba5f251dc7477c8d103b8834ab\": rpc error: code = NotFound desc = could not find container \"f7dff116542e943a60d2eca3094622f0ef67c0ba5f251dc7477c8d103b8834ab\": container with ID starting with f7dff116542e943a60d2eca3094622f0ef67c0ba5f251dc7477c8d103b8834ab not found: ID does not exist"
Sep 19 11:51:00.211066 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:00.211026    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"]
Sep 19 11:51:00.213576 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:00.213555    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-6e19b2-openshift-default-7888649bdb-h95kr"]
Sep 19 11:51:00.595785 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:00.595708    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17fa9957-6197-461e-81a0-e6b305e549c9" path="/var/lib/kubelet/pods/17fa9957-6197-461e-81a0-e6b305e549c9/volumes"
Sep 19 11:51:06.235228 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.235174    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"]
Sep 19 11:51:06.235605 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.235481    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt" podUID="3ffbb330-4619-4940-85f8-576862e1b9d4" containerName="payload-processing" containerID="cri-o://cce14ff69b22362f8933fa0dec21cb1086a442dbf155456b05a0630d0b13d575" gracePeriod=30
Sep 19 11:51:06.243461 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.243426    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"]
Sep 19 11:51:06.243831 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.243652    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms" podUID="10cacae5-7a0b-4f43-9002-d591f799e35c" containerName="payload-pre-processing" containerID="cri-o://699cdbe17c534c47ca38c77c631f322f25d1626279824fc13da9a39b490c1e7b" gracePeriod=30
Sep 19 11:51:06.387169 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.387132    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s"]
Sep 19 11:51:06.390087 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.390053    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-333f49-tjp2s"]
Sep 19 11:51:06.541696 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.541671    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"
Sep 19 11:51:06.544642 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.544624    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"
Sep 19 11:51:06.598175 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.598134    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b28c414b-65da-4e87-96d4-95afb3357103" path="/var/lib/kubelet/pods/b28c414b-65da-4e87-96d4-95afb3357103/volumes"
Sep 19 11:51:06.652029 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.652003    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ffbb330-4619-4940-85f8-576862e1b9d4-plugins-config-volume\") pod \"3ffbb330-4619-4940-85f8-576862e1b9d4\" (UID: \"3ffbb330-4619-4940-85f8-576862e1b9d4\") "
Sep 19 11:51:06.652172 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.652040    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10cacae5-7a0b-4f43-9002-d591f799e35c-plugins-config-volume\") pod \"10cacae5-7a0b-4f43-9002-d591f799e35c\" (UID: \"10cacae5-7a0b-4f43-9002-d591f799e35c\") "
Sep 19 11:51:06.652250 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.652167    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pp42\" (UniqueName: \"kubernetes.io/projected/10cacae5-7a0b-4f43-9002-d591f799e35c-kube-api-access-5pp42\") pod \"10cacae5-7a0b-4f43-9002-d591f799e35c\" (UID: \"10cacae5-7a0b-4f43-9002-d591f799e35c\") "
Sep 19 11:51:06.652297 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.652268    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szwqh\" (UniqueName: \"kubernetes.io/projected/3ffbb330-4619-4940-85f8-576862e1b9d4-kube-api-access-szwqh\") pod \"3ffbb330-4619-4940-85f8-576862e1b9d4\" (UID: \"3ffbb330-4619-4940-85f8-576862e1b9d4\") "
Sep 19 11:51:06.652461 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.652435    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ffbb330-4619-4940-85f8-576862e1b9d4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3ffbb330-4619-4940-85f8-576862e1b9d4" (UID: "3ffbb330-4619-4940-85f8-576862e1b9d4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:51:06.652607 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.652507    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10cacae5-7a0b-4f43-9002-d591f799e35c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "10cacae5-7a0b-4f43-9002-d591f799e35c" (UID: "10cacae5-7a0b-4f43-9002-d591f799e35c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:51:06.652672 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.652648    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10cacae5-7a0b-4f43-9002-d591f799e35c-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:51:06.652672 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.652667    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ffbb330-4619-4940-85f8-576862e1b9d4-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:51:06.654407 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.654382    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ffbb330-4619-4940-85f8-576862e1b9d4-kube-api-access-szwqh" (OuterVolumeSpecName: "kube-api-access-szwqh") pod "3ffbb330-4619-4940-85f8-576862e1b9d4" (UID: "3ffbb330-4619-4940-85f8-576862e1b9d4"). InnerVolumeSpecName "kube-api-access-szwqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:51:06.654511 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.654450    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10cacae5-7a0b-4f43-9002-d591f799e35c-kube-api-access-5pp42" (OuterVolumeSpecName: "kube-api-access-5pp42") pod "10cacae5-7a0b-4f43-9002-d591f799e35c" (UID: "10cacae5-7a0b-4f43-9002-d591f799e35c"). InnerVolumeSpecName "kube-api-access-5pp42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:51:06.753317 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.753286    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szwqh\" (UniqueName: \"kubernetes.io/projected/3ffbb330-4619-4940-85f8-576862e1b9d4-kube-api-access-szwqh\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:51:06.753317 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:06.753311    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pp42\" (UniqueName: \"kubernetes.io/projected/10cacae5-7a0b-4f43-9002-d591f799e35c-kube-api-access-5pp42\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:51:07.235270 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.235233    2559 generic.go:356] "Generic (PLEG): container finished" podID="10cacae5-7a0b-4f43-9002-d591f799e35c" containerID="699cdbe17c534c47ca38c77c631f322f25d1626279824fc13da9a39b490c1e7b" exitCode=0
Sep 19 11:51:07.235797 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.235281    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"
Sep 19 11:51:07.235797 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.235302    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms" event={"ID":"10cacae5-7a0b-4f43-9002-d591f799e35c","Type":"ContainerDied","Data":"699cdbe17c534c47ca38c77c631f322f25d1626279824fc13da9a39b490c1e7b"}
Sep 19 11:51:07.235797 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.235352    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms" event={"ID":"10cacae5-7a0b-4f43-9002-d591f799e35c","Type":"ContainerDied","Data":"3f5b597df8c5516c19a3691ac9e37184424e55ea0b615f150e6077f1b7e8a417"}
Sep 19 11:51:07.235797 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.235381    2559 scope.go:117] "RemoveContainer" containerID="699cdbe17c534c47ca38c77c631f322f25d1626279824fc13da9a39b490c1e7b"
Sep 19 11:51:07.236880 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.236784    2559 generic.go:356] "Generic (PLEG): container finished" podID="3ffbb330-4619-4940-85f8-576862e1b9d4" containerID="cce14ff69b22362f8933fa0dec21cb1086a442dbf155456b05a0630d0b13d575" exitCode=0
Sep 19 11:51:07.236880 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.236832    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt" event={"ID":"3ffbb330-4619-4940-85f8-576862e1b9d4","Type":"ContainerDied","Data":"cce14ff69b22362f8933fa0dec21cb1086a442dbf155456b05a0630d0b13d575"}
Sep 19 11:51:07.236880 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.236850    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"
Sep 19 11:51:07.236880 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.236860    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt" event={"ID":"3ffbb330-4619-4940-85f8-576862e1b9d4","Type":"ContainerDied","Data":"5e6f92e487d6650621e2e231c8a231abec571857feb10e1fba656bb988c599a0"}
Sep 19 11:51:07.248887 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.248870    2559 scope.go:117] "RemoveContainer" containerID="699cdbe17c534c47ca38c77c631f322f25d1626279824fc13da9a39b490c1e7b"
Sep 19 11:51:07.249215 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:51:07.249171    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"699cdbe17c534c47ca38c77c631f322f25d1626279824fc13da9a39b490c1e7b\": container with ID starting with 699cdbe17c534c47ca38c77c631f322f25d1626279824fc13da9a39b490c1e7b not found: ID does not exist" containerID="699cdbe17c534c47ca38c77c631f322f25d1626279824fc13da9a39b490c1e7b"
Sep 19 11:51:07.249278 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.249228    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"699cdbe17c534c47ca38c77c631f322f25d1626279824fc13da9a39b490c1e7b"} err="failed to get container status \"699cdbe17c534c47ca38c77c631f322f25d1626279824fc13da9a39b490c1e7b\": rpc error: code = NotFound desc = could not find container \"699cdbe17c534c47ca38c77c631f322f25d1626279824fc13da9a39b490c1e7b\": container with ID starting with 699cdbe17c534c47ca38c77c631f322f25d1626279824fc13da9a39b490c1e7b not found: ID does not exist"
Sep 19 11:51:07.249278 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.249246    2559 scope.go:117] "RemoveContainer" containerID="cce14ff69b22362f8933fa0dec21cb1086a442dbf155456b05a0630d0b13d575"
Sep 19 11:51:07.263154 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.263133    2559 scope.go:117] "RemoveContainer" containerID="cce14ff69b22362f8933fa0dec21cb1086a442dbf155456b05a0630d0b13d575"
Sep 19 11:51:07.263482 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:51:07.263463    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cce14ff69b22362f8933fa0dec21cb1086a442dbf155456b05a0630d0b13d575\": container with ID starting with cce14ff69b22362f8933fa0dec21cb1086a442dbf155456b05a0630d0b13d575 not found: ID does not exist" containerID="cce14ff69b22362f8933fa0dec21cb1086a442dbf155456b05a0630d0b13d575"
Sep 19 11:51:07.263551 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.263492    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cce14ff69b22362f8933fa0dec21cb1086a442dbf155456b05a0630d0b13d575"} err="failed to get container status \"cce14ff69b22362f8933fa0dec21cb1086a442dbf155456b05a0630d0b13d575\": rpc error: code = NotFound desc = could not find container \"cce14ff69b22362f8933fa0dec21cb1086a442dbf155456b05a0630d0b13d575\": container with ID starting with cce14ff69b22362f8933fa0dec21cb1086a442dbf155456b05a0630d0b13d575 not found: ID does not exist"
Sep 19 11:51:07.271164 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.271141    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"]
Sep 19 11:51:07.275036 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.275014    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-333f49-5c97858fbb-d7vms"]
Sep 19 11:51:07.283093 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.283071    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"]
Sep 19 11:51:07.286428 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:07.286409    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-333f49-6fcff75bc7-svhvt"]
Sep 19 11:51:07.304434 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:51:07.304408    2559 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ffbb330_4619_4940_85f8_576862e1b9d4.slice/crio-5e6f92e487d6650621e2e231c8a231abec571857feb10e1fba656bb988c599a0\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ffbb330_4619_4940_85f8_576862e1b9d4.slice\": RecentStats: unable to find data in memory cache]"
Sep 19 11:51:08.596248 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:08.596214    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10cacae5-7a0b-4f43-9002-d591f799e35c" path="/var/lib/kubelet/pods/10cacae5-7a0b-4f43-9002-d591f799e35c/volumes"
Sep 19 11:51:08.596601 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:08.596565    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ffbb330-4619-4940-85f8-576862e1b9d4" path="/var/lib/kubelet/pods/3ffbb330-4619-4940-85f8-576862e1b9d4/volumes"
Sep 19 11:51:08.653442 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:08.653414    2559 scope.go:117] "RemoveContainer" containerID="5df709d4c07abc29ecf3fedf966b831122935230ef8aa98a3c2a4863eebec540"
Sep 19 11:51:08.662346 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:08.662328    2559 scope.go:117] "RemoveContainer" containerID="2b7e264168658edd264fd38a8fbb5bac9a1086a1708bd8af207d5f403d6d1374"
Sep 19 11:51:08.670481 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:08.670463    2559 scope.go:117] "RemoveContainer" containerID="373b035c74b5aa2d5cd2f11ca19137b0f8a2bed1de7f5ac46d15e70ebdf3b96c"
Sep 19 11:51:13.106047 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.106009    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf"]
Sep 19 11:51:13.106703 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.106683    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b28c414b-65da-4e87-96d4-95afb3357103" containerName="revoke-keys"
Sep 19 11:51:13.106790 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.106707    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b28c414b-65da-4e87-96d4-95afb3357103" containerName="revoke-keys"
Sep 19 11:51:13.106790 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.106722    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17fa9957-6197-461e-81a0-e6b305e549c9" containerName="istio-proxy"
Sep 19 11:51:13.106790 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.106729    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="17fa9957-6197-461e-81a0-e6b305e549c9" containerName="istio-proxy"
Sep 19 11:51:13.106790 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.106758    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ffbb330-4619-4940-85f8-576862e1b9d4" containerName="payload-processing"
Sep 19 11:51:13.106790 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.106770    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ffbb330-4619-4940-85f8-576862e1b9d4" containerName="payload-processing"
Sep 19 11:51:13.106790 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.106787    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10cacae5-7a0b-4f43-9002-d591f799e35c" containerName="payload-pre-processing"
Sep 19 11:51:13.107093 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.106795    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="10cacae5-7a0b-4f43-9002-d591f799e35c" containerName="payload-pre-processing"
Sep 19 11:51:13.107093 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.106895    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="17fa9957-6197-461e-81a0-e6b305e549c9" containerName="istio-proxy"
Sep 19 11:51:13.107093 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.106911    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ffbb330-4619-4940-85f8-576862e1b9d4" containerName="payload-processing"
Sep 19 11:51:13.107093 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.106921    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b28c414b-65da-4e87-96d4-95afb3357103" containerName="revoke-keys"
Sep 19 11:51:13.107093 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.106931    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="10cacae5-7a0b-4f43-9002-d591f799e35c" containerName="payload-pre-processing"
Sep 19 11:51:13.112120 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.112100    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf"
Sep 19 11:51:13.114382 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.114353    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:51:13.116245 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.116220    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf"]
Sep 19 11:51:13.212480 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.212440    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a592392c-2d74-4ae7-b292-d085e7ee55d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf\" (UID: \"a592392c-2d74-4ae7-b292-d085e7ee55d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf"
Sep 19 11:51:13.313439 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.313408    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a592392c-2d74-4ae7-b292-d085e7ee55d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf\" (UID: \"a592392c-2d74-4ae7-b292-d085e7ee55d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf"
Sep 19 11:51:13.314109 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.314076    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a592392c-2d74-4ae7-b292-d085e7ee55d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf\" (UID: \"a592392c-2d74-4ae7-b292-d085e7ee55d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf"
Sep 19 11:51:13.424436 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.424322    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf"
Sep 19 11:51:13.710705 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:13.710679    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf"]
Sep 19 11:51:13.713558 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:51:13.713534    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda592392c_2d74_4ae7_b292_d085e7ee55d0.slice/crio-62ce45c3c24cdc56a27b44276473f343375a6bca5302530153f6947536e75916 WatchSource:0}: Error finding container 62ce45c3c24cdc56a27b44276473f343375a6bca5302530153f6947536e75916: Status 404 returned error can't find the container with id 62ce45c3c24cdc56a27b44276473f343375a6bca5302530153f6947536e75916
Sep 19 11:51:14.281633 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:14.281590    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf" event={"ID":"a592392c-2d74-4ae7-b292-d085e7ee55d0","Type":"ContainerStarted","Data":"cb719eeef53d85e80c8668362014a9db59c234916c77e9b45b146449b1b8efa0"}
Sep 19 11:51:14.282097 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:14.281646    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf" event={"ID":"a592392c-2d74-4ae7-b292-d085e7ee55d0","Type":"ContainerStarted","Data":"62ce45c3c24cdc56a27b44276473f343375a6bca5302530153f6947536e75916"}
Sep 19 11:51:14.298280 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:14.298226    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf" podStartSLOduration=1.298210353 podStartE2EDuration="1.298210353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:51:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:51:14.295744997 +0000 UTC m=+1026.218057878" watchObservedRunningTime="2026-09-19 11:51:14.298210353 +0000 UTC m=+1026.220523234"
Sep 19 11:51:15.289607 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:15.289566    2559 generic.go:356] "Generic (PLEG): container finished" podID="a592392c-2d74-4ae7-b292-d085e7ee55d0" containerID="cb719eeef53d85e80c8668362014a9db59c234916c77e9b45b146449b1b8efa0" exitCode=0
Sep 19 11:51:15.289993 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:15.289611    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf" event={"ID":"a592392c-2d74-4ae7-b292-d085e7ee55d0","Type":"ContainerDied","Data":"cb719eeef53d85e80c8668362014a9db59c234916c77e9b45b146449b1b8efa0"}
Sep 19 11:51:16.435548 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:16.435519    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf"
Sep 19 11:51:16.546406 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:16.546326    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a592392c-2d74-4ae7-b292-d085e7ee55d0-maas-api-service-ca\") pod \"a592392c-2d74-4ae7-b292-d085e7ee55d0\" (UID: \"a592392c-2d74-4ae7-b292-d085e7ee55d0\") "
Sep 19 11:51:16.546693 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:16.546669    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a592392c-2d74-4ae7-b292-d085e7ee55d0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a592392c-2d74-4ae7-b292-d085e7ee55d0" (UID: "a592392c-2d74-4ae7-b292-d085e7ee55d0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:51:16.647916 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:16.647887    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a592392c-2d74-4ae7-b292-d085e7ee55d0-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:51:17.302003 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:17.301975    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf"
Sep 19 11:51:17.302003 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:17.301991    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf" event={"ID":"a592392c-2d74-4ae7-b292-d085e7ee55d0","Type":"ContainerDied","Data":"62ce45c3c24cdc56a27b44276473f343375a6bca5302530153f6947536e75916"}
Sep 19 11:51:17.302243 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:17.302037    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62ce45c3c24cdc56a27b44276473f343375a6bca5302530153f6947536e75916"
Sep 19 11:51:23.684768 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:23.684731    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d8d8bb-29830305-778hr"]
Sep 19 11:51:23.695891 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:23.695854    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d8d8bb-29830305-778hr"]
Sep 19 11:51:23.713049 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:23.713012    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"]
Sep 19 11:51:23.713348 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:23.713302    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c" podUID="c54f9900-9aaf-4ac7-990d-d32b428e1a96" containerName="payload-processing" containerID="cri-o://9d6102d64c66893c3869807af4b55196556080188cbd3b337e5b54adc5088aa4" gracePeriod=30
Sep 19 11:51:23.720817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:23.720790    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"]
Sep 19 11:51:23.721176 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:23.721132    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg" podUID="a003bb0b-8221-4933-ab41-8fd3fc7ed5a5" containerName="payload-pre-processing" containerID="cri-o://11c7d078b1f0c5db872fc8229f46a59a35fa55fb566edea0f62a8e29c422b52b" gracePeriod=30
Sep 19 11:51:23.948016 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:23.945394    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf"]
Sep 19 11:51:23.955052 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:23.950444    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d8d8bb-dsmrf"]
Sep 19 11:51:24.014580 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.014557    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"
Sep 19 11:51:24.017751 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.017726    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"
Sep 19 11:51:24.120705 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.120673    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a003bb0b-8221-4933-ab41-8fd3fc7ed5a5-plugins-config-volume\") pod \"a003bb0b-8221-4933-ab41-8fd3fc7ed5a5\" (UID: \"a003bb0b-8221-4933-ab41-8fd3fc7ed5a5\") "
Sep 19 11:51:24.120870 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.120811    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c54f9900-9aaf-4ac7-990d-d32b428e1a96-plugins-config-volume\") pod \"c54f9900-9aaf-4ac7-990d-d32b428e1a96\" (UID: \"c54f9900-9aaf-4ac7-990d-d32b428e1a96\") "
Sep 19 11:51:24.120870 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.120828    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxrqc\" (UniqueName: \"kubernetes.io/projected/c54f9900-9aaf-4ac7-990d-d32b428e1a96-kube-api-access-hxrqc\") pod \"c54f9900-9aaf-4ac7-990d-d32b428e1a96\" (UID: \"c54f9900-9aaf-4ac7-990d-d32b428e1a96\") "
Sep 19 11:51:24.120870 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.120857    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8c4l8\" (UniqueName: \"kubernetes.io/projected/a003bb0b-8221-4933-ab41-8fd3fc7ed5a5-kube-api-access-8c4l8\") pod \"a003bb0b-8221-4933-ab41-8fd3fc7ed5a5\" (UID: \"a003bb0b-8221-4933-ab41-8fd3fc7ed5a5\") "
Sep 19 11:51:24.121129 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.121099    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a003bb0b-8221-4933-ab41-8fd3fc7ed5a5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a003bb0b-8221-4933-ab41-8fd3fc7ed5a5" (UID: "a003bb0b-8221-4933-ab41-8fd3fc7ed5a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:51:24.121182 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.121141    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c54f9900-9aaf-4ac7-990d-d32b428e1a96-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c54f9900-9aaf-4ac7-990d-d32b428e1a96" (UID: "c54f9900-9aaf-4ac7-990d-d32b428e1a96"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:51:24.123073 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.123050    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a003bb0b-8221-4933-ab41-8fd3fc7ed5a5-kube-api-access-8c4l8" (OuterVolumeSpecName: "kube-api-access-8c4l8") pod "a003bb0b-8221-4933-ab41-8fd3fc7ed5a5" (UID: "a003bb0b-8221-4933-ab41-8fd3fc7ed5a5"). InnerVolumeSpecName "kube-api-access-8c4l8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:51:24.123178 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.123157    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c54f9900-9aaf-4ac7-990d-d32b428e1a96-kube-api-access-hxrqc" (OuterVolumeSpecName: "kube-api-access-hxrqc") pod "c54f9900-9aaf-4ac7-990d-d32b428e1a96" (UID: "c54f9900-9aaf-4ac7-990d-d32b428e1a96"). InnerVolumeSpecName "kube-api-access-hxrqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:51:24.221915 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.221840    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c54f9900-9aaf-4ac7-990d-d32b428e1a96-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:51:24.221915 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.221867    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxrqc\" (UniqueName: \"kubernetes.io/projected/c54f9900-9aaf-4ac7-990d-d32b428e1a96-kube-api-access-hxrqc\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:51:24.221915 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.221880    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8c4l8\" (UniqueName: \"kubernetes.io/projected/a003bb0b-8221-4933-ab41-8fd3fc7ed5a5-kube-api-access-8c4l8\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:51:24.221915 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.221891    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a003bb0b-8221-4933-ab41-8fd3fc7ed5a5-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:51:24.341429 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.341400    2559 generic.go:356] "Generic (PLEG): container finished" podID="c54f9900-9aaf-4ac7-990d-d32b428e1a96" containerID="9d6102d64c66893c3869807af4b55196556080188cbd3b337e5b54adc5088aa4" exitCode=0
Sep 19 11:51:24.341583 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.341442    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"
Sep 19 11:51:24.341583 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.341441    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c" event={"ID":"c54f9900-9aaf-4ac7-990d-d32b428e1a96","Type":"ContainerDied","Data":"9d6102d64c66893c3869807af4b55196556080188cbd3b337e5b54adc5088aa4"}
Sep 19 11:51:24.341583 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.341507    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c" event={"ID":"c54f9900-9aaf-4ac7-990d-d32b428e1a96","Type":"ContainerDied","Data":"77219c54e1b09c3577ca210127059b64d5d8c040953459cc04fe58eb7c17d920"}
Sep 19 11:51:24.341583 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.341531    2559 scope.go:117] "RemoveContainer" containerID="9d6102d64c66893c3869807af4b55196556080188cbd3b337e5b54adc5088aa4"
Sep 19 11:51:24.343078 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.343055    2559 generic.go:356] "Generic (PLEG): container finished" podID="a003bb0b-8221-4933-ab41-8fd3fc7ed5a5" containerID="11c7d078b1f0c5db872fc8229f46a59a35fa55fb566edea0f62a8e29c422b52b" exitCode=0
Sep 19 11:51:24.343218 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.343077    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg" event={"ID":"a003bb0b-8221-4933-ab41-8fd3fc7ed5a5","Type":"ContainerDied","Data":"11c7d078b1f0c5db872fc8229f46a59a35fa55fb566edea0f62a8e29c422b52b"}
Sep 19 11:51:24.343218 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.343112    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"
Sep 19 11:51:24.343218 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.343128    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg" event={"ID":"a003bb0b-8221-4933-ab41-8fd3fc7ed5a5","Type":"ContainerDied","Data":"1cc9aef6c258f101dae2d3ef7183ee993c9f31a9063e960b64e1c0173e3c227d"}
Sep 19 11:51:24.351469 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.351455    2559 scope.go:117] "RemoveContainer" containerID="9d6102d64c66893c3869807af4b55196556080188cbd3b337e5b54adc5088aa4"
Sep 19 11:51:24.351773 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:51:24.351744    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d6102d64c66893c3869807af4b55196556080188cbd3b337e5b54adc5088aa4\": container with ID starting with 9d6102d64c66893c3869807af4b55196556080188cbd3b337e5b54adc5088aa4 not found: ID does not exist" containerID="9d6102d64c66893c3869807af4b55196556080188cbd3b337e5b54adc5088aa4"
Sep 19 11:51:24.351871 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.351782    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d6102d64c66893c3869807af4b55196556080188cbd3b337e5b54adc5088aa4"} err="failed to get container status \"9d6102d64c66893c3869807af4b55196556080188cbd3b337e5b54adc5088aa4\": rpc error: code = NotFound desc = could not find container \"9d6102d64c66893c3869807af4b55196556080188cbd3b337e5b54adc5088aa4\": container with ID starting with 9d6102d64c66893c3869807af4b55196556080188cbd3b337e5b54adc5088aa4 not found: ID does not exist"
Sep 19 11:51:24.351871 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.351805    2559 scope.go:117] "RemoveContainer" containerID="11c7d078b1f0c5db872fc8229f46a59a35fa55fb566edea0f62a8e29c422b52b"
Sep 19 11:51:24.360029 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.360013    2559 scope.go:117] "RemoveContainer" containerID="11c7d078b1f0c5db872fc8229f46a59a35fa55fb566edea0f62a8e29c422b52b"
Sep 19 11:51:24.360344 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:51:24.360318    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11c7d078b1f0c5db872fc8229f46a59a35fa55fb566edea0f62a8e29c422b52b\": container with ID starting with 11c7d078b1f0c5db872fc8229f46a59a35fa55fb566edea0f62a8e29c422b52b not found: ID does not exist" containerID="11c7d078b1f0c5db872fc8229f46a59a35fa55fb566edea0f62a8e29c422b52b"
Sep 19 11:51:24.360414 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.360344    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11c7d078b1f0c5db872fc8229f46a59a35fa55fb566edea0f62a8e29c422b52b"} err="failed to get container status \"11c7d078b1f0c5db872fc8229f46a59a35fa55fb566edea0f62a8e29c422b52b\": rpc error: code = NotFound desc = could not find container \"11c7d078b1f0c5db872fc8229f46a59a35fa55fb566edea0f62a8e29c422b52b\": container with ID starting with 11c7d078b1f0c5db872fc8229f46a59a35fa55fb566edea0f62a8e29c422b52b not found: ID does not exist"
Sep 19 11:51:24.363753 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.363731    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"]
Sep 19 11:51:24.367353 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.367330    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d8d8bb-7d75546575-2sg4c"]
Sep 19 11:51:24.377467 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.377446    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"]
Sep 19 11:51:24.380786 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.380765    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d8d8bb-5c85569bc4-5cqfg"]
Sep 19 11:51:24.596474 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.596390    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c27a66b-d680-4f5d-8d28-6b632e69c131" path="/var/lib/kubelet/pods/4c27a66b-d680-4f5d-8d28-6b632e69c131/volumes"
Sep 19 11:51:24.596817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.596798    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a003bb0b-8221-4933-ab41-8fd3fc7ed5a5" path="/var/lib/kubelet/pods/a003bb0b-8221-4933-ab41-8fd3fc7ed5a5/volumes"
Sep 19 11:51:24.597158 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.597139    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a592392c-2d74-4ae7-b292-d085e7ee55d0" path="/var/lib/kubelet/pods/a592392c-2d74-4ae7-b292-d085e7ee55d0/volumes"
Sep 19 11:51:24.597559 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:24.597541    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c54f9900-9aaf-4ac7-990d-d32b428e1a96" path="/var/lib/kubelet/pods/c54f9900-9aaf-4ac7-990d-d32b428e1a96/volumes"
Sep 19 11:51:31.003353 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.003326    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"]
Sep 19 11:51:31.003817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.003763    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a003bb0b-8221-4933-ab41-8fd3fc7ed5a5" containerName="payload-pre-processing"
Sep 19 11:51:31.003817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.003775    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a003bb0b-8221-4933-ab41-8fd3fc7ed5a5" containerName="payload-pre-processing"
Sep 19 11:51:31.003817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.003790    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54f9900-9aaf-4ac7-990d-d32b428e1a96" containerName="payload-processing"
Sep 19 11:51:31.003817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.003797    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54f9900-9aaf-4ac7-990d-d32b428e1a96" containerName="payload-processing"
Sep 19 11:51:31.003817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.003805    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a592392c-2d74-4ae7-b292-d085e7ee55d0" containerName="revoke-keys"
Sep 19 11:51:31.003817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.003811    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a592392c-2d74-4ae7-b292-d085e7ee55d0" containerName="revoke-keys"
Sep 19 11:51:31.004040 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.003878    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54f9900-9aaf-4ac7-990d-d32b428e1a96" containerName="payload-processing"
Sep 19 11:51:31.004040 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.003888    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a592392c-2d74-4ae7-b292-d085e7ee55d0" containerName="revoke-keys"
Sep 19 11:51:31.004040 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.003896    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a003bb0b-8221-4933-ab41-8fd3fc7ed5a5" containerName="payload-pre-processing"
Sep 19 11:51:31.008497 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.008480    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.010740 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.010719    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-4937ff-openshift-default-dockercfg-cxqlb\""
Sep 19 11:51:31.015874 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.015855    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"]
Sep 19 11:51:31.086147 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.086101    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v6cb\" (UniqueName: \"kubernetes.io/projected/4afed85b-980b-4c82-b7ea-894e3aee992a-kube-api-access-7v6cb\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.086327 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.086155    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-workload-socket\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.086327 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.086184    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-envoy\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.086327 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.086247    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-credential-socket\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.086327 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.086301    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-podinfo\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.086327 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.086321    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-token\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.086546 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.086382    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-workload-certs\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.086546 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.086405    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-data\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.086546 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.086437    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4afed85b-980b-4c82-b7ea-894e3aee992a-istiod-ca-cert\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.187520 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.187486    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v6cb\" (UniqueName: \"kubernetes.io/projected/4afed85b-980b-4c82-b7ea-894e3aee992a-kube-api-access-7v6cb\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.187520 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.187524    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-workload-socket\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.187771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.187542    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-envoy\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.187771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.187570    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-credential-socket\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.187771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.187615    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-podinfo\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.187771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.187635    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-token\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.187771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.187680    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-workload-certs\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.187771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.187707    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-data\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.187771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.187745    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4afed85b-980b-4c82-b7ea-894e3aee992a-istiod-ca-cert\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.188163 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.187954    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-workload-socket\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.188163 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.188003    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-credential-socket\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.188163 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.188072    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-workload-certs\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.188163 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.188128    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-data\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.188415 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.188398    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4afed85b-980b-4c82-b7ea-894e3aee992a-istiod-ca-cert\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.190543 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.190515    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-envoy\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.190669 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.190574    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-podinfo\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.195208 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.195163    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-token\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.195327 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.195311    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v6cb\" (UniqueName: \"kubernetes.io/projected/4afed85b-980b-4c82-b7ea-894e3aee992a-kube-api-access-7v6cb\") pod \"e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") " pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.320714 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.320630    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:31.595572 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.595547    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"]
Sep 19 11:51:31.597219 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:51:31.597158    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4afed85b_980b_4c82_b7ea_894e3aee992a.slice/crio-e64377258aeddce3a60ebbff93fed0fe743ddc159213c413ec923aeefb7c1227 WatchSource:0}: Error finding container e64377258aeddce3a60ebbff93fed0fe743ddc159213c413ec923aeefb7c1227: Status 404 returned error can't find the container with id e64377258aeddce3a60ebbff93fed0fe743ddc159213c413ec923aeefb7c1227
Sep 19 11:51:31.599344 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.599308    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:51:31.599437 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.599372    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:51:31.599437 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:31.599401    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:51:32.397179 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:32.397148    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x" event={"ID":"4afed85b-980b-4c82-b7ea-894e3aee992a","Type":"ContainerStarted","Data":"e8f9a8ab29aa9b37013cac4cb35c41ddc7b546bbf70504634fcc0a5d0d7bb00a"}
Sep 19 11:51:32.397650 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:32.397246    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x" event={"ID":"4afed85b-980b-4c82-b7ea-894e3aee992a","Type":"ContainerStarted","Data":"e64377258aeddce3a60ebbff93fed0fe743ddc159213c413ec923aeefb7c1227"}
Sep 19 11:51:32.415986 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:32.415942    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x" podStartSLOduration=2.41592456 podStartE2EDuration="2.41592456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:51:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:51:32.415161214 +0000 UTC m=+1044.337474095" watchObservedRunningTime="2026-09-19 11:51:32.41592456 +0000 UTC m=+1044.338237442"
Sep 19 11:51:33.321539 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:33.321500    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:33.326758 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:33.326734    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:33.403107 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:33.403077    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:33.403897 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:33.403877    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:51:37.902500 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:37.902465    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"]
Sep 19 11:51:37.906095 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:37.906074    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"
Sep 19 11:51:37.908379 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:37.908358    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-4937ff-dockercfg-kqbf4\""
Sep 19 11:51:37.908482 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:37.908408    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-4937ff\""
Sep 19 11:51:37.915949 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:37.915922    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"]
Sep 19 11:51:37.957579 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:37.957552    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlcrj\" (UniqueName: \"kubernetes.io/projected/3c12cd77-c733-48ed-9a3f-71db94cc08fc-kube-api-access-dlcrj\") pod \"payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p\" (UID: \"3c12cd77-c733-48ed-9a3f-71db94cc08fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"
Sep 19 11:51:37.957745 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:37.957593    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c12cd77-c733-48ed-9a3f-71db94cc08fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p\" (UID: \"3c12cd77-c733-48ed-9a3f-71db94cc08fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"
Sep 19 11:51:38.058977 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:38.058935    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c12cd77-c733-48ed-9a3f-71db94cc08fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p\" (UID: \"3c12cd77-c733-48ed-9a3f-71db94cc08fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"
Sep 19 11:51:38.059168 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:38.059058    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlcrj\" (UniqueName: \"kubernetes.io/projected/3c12cd77-c733-48ed-9a3f-71db94cc08fc-kube-api-access-dlcrj\") pod \"payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p\" (UID: \"3c12cd77-c733-48ed-9a3f-71db94cc08fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"
Sep 19 11:51:38.059730 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:38.059706    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c12cd77-c733-48ed-9a3f-71db94cc08fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p\" (UID: \"3c12cd77-c733-48ed-9a3f-71db94cc08fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"
Sep 19 11:51:38.066595 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:38.066566    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlcrj\" (UniqueName: \"kubernetes.io/projected/3c12cd77-c733-48ed-9a3f-71db94cc08fc-kube-api-access-dlcrj\") pod \"payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p\" (UID: \"3c12cd77-c733-48ed-9a3f-71db94cc08fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"
Sep 19 11:51:38.218448 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:38.218416    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"
Sep 19 11:51:38.492040 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:38.492008    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"]
Sep 19 11:51:38.492987 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:51:38.492958    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c12cd77_c733_48ed_9a3f_71db94cc08fc.slice/crio-e6d0e081251c37d7679c72c70d60d17a56299035ba51517ebaa2d8cd918097a6 WatchSource:0}: Error finding container e6d0e081251c37d7679c72c70d60d17a56299035ba51517ebaa2d8cd918097a6: Status 404 returned error can't find the container with id e6d0e081251c37d7679c72c70d60d17a56299035ba51517ebaa2d8cd918097a6
Sep 19 11:51:39.440845 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:39.440813    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p" event={"ID":"3c12cd77-c733-48ed-9a3f-71db94cc08fc","Type":"ContainerStarted","Data":"dee3390ddf741b8d0ad7d9c7818f13688af745c822cd486736d3de1a97b2adf8"}
Sep 19 11:51:39.441308 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:39.440862    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p" event={"ID":"3c12cd77-c733-48ed-9a3f-71db94cc08fc","Type":"ContainerStarted","Data":"e6d0e081251c37d7679c72c70d60d17a56299035ba51517ebaa2d8cd918097a6"}
Sep 19 11:51:39.441308 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:39.440913    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"
Sep 19 11:51:39.458687 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:39.458640    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p" podStartSLOduration=2.458626978 podStartE2EDuration="2.458626978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:51:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:51:39.45630803 +0000 UTC m=+1051.378620926" watchObservedRunningTime="2026-09-19 11:51:39.458626978 +0000 UTC m=+1051.380939859"
Sep 19 11:51:50.449040 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:50.449007    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"
Sep 19 11:51:59.585112 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.585078    2559 generic.go:356] "Generic (PLEG): container finished" podID="4afed85b-980b-4c82-b7ea-894e3aee992a" containerID="e8f9a8ab29aa9b37013cac4cb35c41ddc7b546bbf70504634fcc0a5d0d7bb00a" exitCode=0
Sep 19 11:51:59.585585 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.585158    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x" event={"ID":"4afed85b-980b-4c82-b7ea-894e3aee992a","Type":"ContainerDied","Data":"e8f9a8ab29aa9b37013cac4cb35c41ddc7b546bbf70504634fcc0a5d0d7bb00a"}
Sep 19 11:51:59.585734 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.585718    2559 scope.go:117] "RemoveContainer" containerID="e8f9a8ab29aa9b37013cac4cb35c41ddc7b546bbf70504634fcc0a5d0d7bb00a"
Sep 19 11:51:59.587113 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.587080    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:51:59.587241 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.587164    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:51:59.587321 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.587247    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:51:59.876797 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.876719    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"]
Sep 19 11:51:59.887272 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.887231    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:51:59.892044 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.892016    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-4a8a59-openshift-default-dockercfg-lnnkd\""
Sep 19 11:51:59.895206 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.894988    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"]
Sep 19 11:51:59.973244 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.973163    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-credential-socket\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:51:59.973421 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.973299    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-workload-certs\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:51:59.973421 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.973336    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7672\" (UniqueName: \"kubernetes.io/projected/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-kube-api-access-k7672\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:51:59.973564 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.973424    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-token\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:51:59.973564 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.973480    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-podinfo\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:51:59.973564 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.973520    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-workload-socket\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:51:59.973564 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.973551    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istiod-ca-cert\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:51:59.973775 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.973598    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-envoy\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:51:59.973775 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:51:59.973660    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-data\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.075035 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.074998    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-data\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.075182 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.075053    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-credential-socket\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.075182 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.075092    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-workload-certs\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.075182 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.075119    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7672\" (UniqueName: \"kubernetes.io/projected/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-kube-api-access-k7672\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.075182 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.075177    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-token\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.075431 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.075246    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-podinfo\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.075431 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.075275    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-workload-socket\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.075431 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.075306    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istiod-ca-cert\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.075431 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.075353    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-envoy\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.075643 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.075481    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-data\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.075643 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.075537    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-credential-socket\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.075643 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.075582    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-workload-socket\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.075805 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.075740    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-workload-certs\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.075998 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.075980    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istiod-ca-cert\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.077591 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.077572    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-envoy\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.077858 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.077837    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-podinfo\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.083033 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.083003    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-token\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.083612 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.083592    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7672\" (UniqueName: \"kubernetes.io/projected/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-kube-api-access-k7672\") pod \"e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") " pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.204592 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.204554    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:00.486068 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:52:00.486039    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b729bce_b9dc_4c2a_8ad7_1ad68b68dc85.slice/crio-2c38d6d57c4a857d1ae4895eea630a7cbca00b4b1ca68ac000945b78ca986da1 WatchSource:0}: Error finding container 2c38d6d57c4a857d1ae4895eea630a7cbca00b4b1ca68ac000945b78ca986da1: Status 404 returned error can't find the container with id 2c38d6d57c4a857d1ae4895eea630a7cbca00b4b1ca68ac000945b78ca986da1
Sep 19 11:52:00.486433 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.486411    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"]
Sep 19 11:52:00.488452 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.488419    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:52:00.488564 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.488483    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:52:00.488564 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.488513    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:52:00.610214 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.609618    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl" event={"ID":"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85","Type":"ContainerStarted","Data":"ad763b6097f66b6838d3af3255275e859f8c885eee504ecb13d71bb4732c29bc"}
Sep 19 11:52:00.610214 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.609698    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl" event={"ID":"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85","Type":"ContainerStarted","Data":"2c38d6d57c4a857d1ae4895eea630a7cbca00b4b1ca68ac000945b78ca986da1"}
Sep 19 11:52:00.612221 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.611660    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x" event={"ID":"4afed85b-980b-4c82-b7ea-894e3aee992a","Type":"ContainerStarted","Data":"85969c55fca49fd2cac55711384fe8973013012eb5a219b73137cfa4a43a6fb1"}
Sep 19 11:52:00.629128 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:00.629075    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl" podStartSLOduration=1.6290545440000002 podStartE2EDuration="1.629054544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:51:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:52:00.626070153 +0000 UTC m=+1072.548383061" watchObservedRunningTime="2026-09-19 11:52:00.629054544 +0000 UTC m=+1072.551367427"
Sep 19 11:52:01.205901 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:01.205866    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:01.211162 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:01.211136    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:01.321097 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:01.321061    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:52:01.322266 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:01.322245    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 19 11:52:01.322374 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:01.322290    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x" podUID="4afed85b-980b-4c82-b7ea-894e3aee992a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 19 11:52:01.617322 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:01.617227    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:01.618179 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:01.618163    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:02.321093 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:02.321056    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 19 11:52:02.321304 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:02.321115    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x" podUID="4afed85b-980b-4c82-b7ea-894e3aee992a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 19 11:52:03.321114 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:03.321072    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 19 11:52:03.321606 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:03.321157    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x" podUID="4afed85b-980b-4c82-b7ea-894e3aee992a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 19 11:52:03.403662 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:03.403609    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:52:04.321291 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:04.321254    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 19 11:52:04.321705 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:04.321349    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x" podUID="4afed85b-980b-4c82-b7ea-894e3aee992a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 19 11:52:04.624114 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:04.624043    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"]
Sep 19 11:52:04.627716 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:04.627696    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"
Sep 19 11:52:04.630138 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:04.630082    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-4a8a59-dockercfg-gh76n\""
Sep 19 11:52:04.630138 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:04.630094    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-4a8a59\""
Sep 19 11:52:04.639046 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:04.639015    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"]
Sep 19 11:52:04.732895 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:04.732860    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8br4p\" (UniqueName: \"kubernetes.io/projected/76519908-fac0-4508-b417-78423417cf34-kube-api-access-8br4p\") pod \"payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft\" (UID: \"76519908-fac0-4508-b417-78423417cf34\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"
Sep 19 11:52:04.733076 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:04.733065    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76519908-fac0-4508-b417-78423417cf34-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft\" (UID: \"76519908-fac0-4508-b417-78423417cf34\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"
Sep 19 11:52:04.833922 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:04.833889    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8br4p\" (UniqueName: \"kubernetes.io/projected/76519908-fac0-4508-b417-78423417cf34-kube-api-access-8br4p\") pod \"payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft\" (UID: \"76519908-fac0-4508-b417-78423417cf34\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"
Sep 19 11:52:04.834098 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:04.833929    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76519908-fac0-4508-b417-78423417cf34-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft\" (UID: \"76519908-fac0-4508-b417-78423417cf34\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"
Sep 19 11:52:04.834731 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:04.834707    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76519908-fac0-4508-b417-78423417cf34-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft\" (UID: \"76519908-fac0-4508-b417-78423417cf34\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"
Sep 19 11:52:04.842051 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:04.842031    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8br4p\" (UniqueName: \"kubernetes.io/projected/76519908-fac0-4508-b417-78423417cf34-kube-api-access-8br4p\") pod \"payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft\" (UID: \"76519908-fac0-4508-b417-78423417cf34\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"
Sep 19 11:52:04.940753 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:04.940731    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"
Sep 19 11:52:05.235859 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:05.235834    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"]
Sep 19 11:52:05.237785 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:52:05.237746    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76519908_fac0_4508_b417_78423417cf34.slice/crio-dbd32cbeffbbfd51ba6ba6713db877deb9df9282ceec10e9862d3f478f8c8125 WatchSource:0}: Error finding container dbd32cbeffbbfd51ba6ba6713db877deb9df9282ceec10e9862d3f478f8c8125: Status 404 returned error can't find the container with id dbd32cbeffbbfd51ba6ba6713db877deb9df9282ceec10e9862d3f478f8c8125
Sep 19 11:52:05.321667 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:05.321638    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Sep 19 11:52:05.322073 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:05.321694    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x" podUID="4afed85b-980b-4c82-b7ea-894e3aee992a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Sep 19 11:52:05.644057 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:05.643969    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft" event={"ID":"76519908-fac0-4508-b417-78423417cf34","Type":"ContainerStarted","Data":"c710b5837e9ff144100671de7fac21f44ea9eb55e9cbc2389adce7b8565e1785"}
Sep 19 11:52:05.644057 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:05.644029    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft" event={"ID":"76519908-fac0-4508-b417-78423417cf34","Type":"ContainerStarted","Data":"dbd32cbeffbbfd51ba6ba6713db877deb9df9282ceec10e9862d3f478f8c8125"}
Sep 19 11:52:05.644288 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:05.644152    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"
Sep 19 11:52:05.659065 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:05.659021    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft" podStartSLOduration=1.659003378 podStartE2EDuration="1.659003378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:52:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:52:05.657089379 +0000 UTC m=+1077.579402261" watchObservedRunningTime="2026-09-19 11:52:05.659003378 +0000 UTC m=+1077.581316261"
Sep 19 11:52:06.325457 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:06.325426    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:52:06.326535 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:06.326516    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 11:52:08.710704 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:08.710675    2559 scope.go:117] "RemoveContainer" containerID="e239220aecf16d48f18a0c8370df89d5b77fd8000a84c71e7386adef09305e64"
Sep 19 11:52:08.719518 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:08.719502    2559 scope.go:117] "RemoveContainer" containerID="12a98a386dbbf47746c93172d19d2b27dca3b90215d8e37cbb763deb0c9288d3"
Sep 19 11:52:16.652286 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:16.652258    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"
Sep 19 11:52:29.795754 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:29.795714    2559 generic.go:356] "Generic (PLEG): container finished" podID="6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" containerID="ad763b6097f66b6838d3af3255275e859f8c885eee504ecb13d71bb4732c29bc" exitCode=0
Sep 19 11:52:29.796291 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:29.795786    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl" event={"ID":"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85","Type":"ContainerDied","Data":"ad763b6097f66b6838d3af3255275e859f8c885eee504ecb13d71bb4732c29bc"}
Sep 19 11:52:29.796382 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:29.796324    2559 scope.go:117] "RemoveContainer" containerID="ad763b6097f66b6838d3af3255275e859f8c885eee504ecb13d71bb4732c29bc"
Sep 19 11:52:29.797623 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:29.797590    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:52:29.797738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:29.797674    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:52:29.797738 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:29.797724    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:52:30.807780 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:30.807738    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl" event={"ID":"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85","Type":"ContainerStarted","Data":"9290649dd999dbcc3a55413835a677c7b4b0885da4d1227c19c2086069e9b283"}
Sep 19 11:52:31.205631 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:31.205594    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:31.206779 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:31.206750    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 19 11:52:31.206906 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:31.206809    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl" podUID="6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 19 11:52:31.617208 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:31.617109    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:32.205807 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:32.205768    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 19 11:52:32.206441 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:32.206399    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl" podUID="6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 19 11:52:33.205893 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:33.205863    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 19 11:52:33.206354 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:33.206312    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl" podUID="6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 19 11:52:34.205651 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:34.205616    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 19 11:52:34.205954 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:34.205925    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl" podUID="6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 19 11:52:35.205020 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:35.204973    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 19 11:52:35.205263 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:35.205041    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl" podUID="6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 19 11:52:36.205089 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:36.205046    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 19 11:52:36.205517 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:36.205110    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl" podUID="6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 19 11:52:37.209975 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:37.209942    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:37.210857 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:37.210841    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 11:52:47.418548 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:47.418512    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4"]
Sep 19 11:52:47.422368 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:47.422350    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4"
Sep 19 11:52:47.425225 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:47.425207    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:52:47.439667 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:47.439641    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4"]
Sep 19 11:52:47.459945 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:47.459916    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05a402a6-e5b8-4fc0-84cc-214f443ffbfd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4\" (UID: \"05a402a6-e5b8-4fc0-84cc-214f443ffbfd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4"
Sep 19 11:52:47.560681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:47.560647    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05a402a6-e5b8-4fc0-84cc-214f443ffbfd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4\" (UID: \"05a402a6-e5b8-4fc0-84cc-214f443ffbfd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4"
Sep 19 11:52:47.561326 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:47.561306    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05a402a6-e5b8-4fc0-84cc-214f443ffbfd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4\" (UID: \"05a402a6-e5b8-4fc0-84cc-214f443ffbfd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4"
Sep 19 11:52:47.733088 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:47.733052    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4"
Sep 19 11:52:48.003437 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:48.003410    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4"]
Sep 19 11:52:48.005244 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:52:48.005216    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05a402a6_e5b8_4fc0_84cc_214f443ffbfd.slice/crio-804545f8970bcd3400e12692372c3d59f91c5754856482d0e129dfa36cd3eb89 WatchSource:0}: Error finding container 804545f8970bcd3400e12692372c3d59f91c5754856482d0e129dfa36cd3eb89: Status 404 returned error can't find the container with id 804545f8970bcd3400e12692372c3d59f91c5754856482d0e129dfa36cd3eb89
Sep 19 11:52:48.914638 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:48.914604    2559 generic.go:356] "Generic (PLEG): container finished" podID="05a402a6-e5b8-4fc0-84cc-214f443ffbfd" containerID="3051760befd4e6004e68b25c006d61b4d1babb105c87efb4be731da88752c8ec" exitCode=0
Sep 19 11:52:48.915000 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:48.914689    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4" event={"ID":"05a402a6-e5b8-4fc0-84cc-214f443ffbfd","Type":"ContainerDied","Data":"3051760befd4e6004e68b25c006d61b4d1babb105c87efb4be731da88752c8ec"}
Sep 19 11:52:48.915000 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:48.914740    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4" event={"ID":"05a402a6-e5b8-4fc0-84cc-214f443ffbfd","Type":"ContainerStarted","Data":"804545f8970bcd3400e12692372c3d59f91c5754856482d0e129dfa36cd3eb89"}
Sep 19 11:52:50.070881 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:50.070857    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4"
Sep 19 11:52:50.188350 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:50.188325    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05a402a6-e5b8-4fc0-84cc-214f443ffbfd-maas-api-service-ca\") pod \"05a402a6-e5b8-4fc0-84cc-214f443ffbfd\" (UID: \"05a402a6-e5b8-4fc0-84cc-214f443ffbfd\") "
Sep 19 11:52:50.188548 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:50.188527    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05a402a6-e5b8-4fc0-84cc-214f443ffbfd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "05a402a6-e5b8-4fc0-84cc-214f443ffbfd" (UID: "05a402a6-e5b8-4fc0-84cc-214f443ffbfd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:52:50.188694 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:50.188682    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05a402a6-e5b8-4fc0-84cc-214f443ffbfd-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:52:50.928762 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:50.928683    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4"
Sep 19 11:52:50.928762 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:50.928695    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4" event={"ID":"05a402a6-e5b8-4fc0-84cc-214f443ffbfd","Type":"ContainerDied","Data":"804545f8970bcd3400e12692372c3d59f91c5754856482d0e129dfa36cd3eb89"}
Sep 19 11:52:50.928762 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:52:50.928748    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="804545f8970bcd3400e12692372c3d59f91c5754856482d0e129dfa36cd3eb89"
Sep 19 11:53:02.905752 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:02.905713    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"]
Sep 19 11:53:02.906237 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:02.905987    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5" podUID="1d01315b-0855-433a-8e81-337384350872" containerName="payload-processing" containerID="cri-o://6f3f37d79d3b3cd8548d398653553da10af93eb880c263b552ef88f14a3e463b" gracePeriod=30
Sep 19 11:53:03.062611 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:03.062576    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4"]
Sep 19 11:53:03.067002 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:03.066973    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcfc8fcc-k8mk4"]
Sep 19 11:53:03.189225 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:03.189183    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"
Sep 19 11:53:03.331091 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:03.331060    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d01315b-0855-433a-8e81-337384350872-plugins-config-volume\") pod \"1d01315b-0855-433a-8e81-337384350872\" (UID: \"1d01315b-0855-433a-8e81-337384350872\") "
Sep 19 11:53:03.331307 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:03.331184    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8b7x6\" (UniqueName: \"kubernetes.io/projected/1d01315b-0855-433a-8e81-337384350872-kube-api-access-8b7x6\") pod \"1d01315b-0855-433a-8e81-337384350872\" (UID: \"1d01315b-0855-433a-8e81-337384350872\") "
Sep 19 11:53:03.331456 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:03.331431    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d01315b-0855-433a-8e81-337384350872-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1d01315b-0855-433a-8e81-337384350872" (UID: "1d01315b-0855-433a-8e81-337384350872"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:53:03.333214 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:03.333171    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d01315b-0855-433a-8e81-337384350872-kube-api-access-8b7x6" (OuterVolumeSpecName: "kube-api-access-8b7x6") pod "1d01315b-0855-433a-8e81-337384350872" (UID: "1d01315b-0855-433a-8e81-337384350872"). InnerVolumeSpecName "kube-api-access-8b7x6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:53:03.432394 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:03.432364    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8b7x6\" (UniqueName: \"kubernetes.io/projected/1d01315b-0855-433a-8e81-337384350872-kube-api-access-8b7x6\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:53:03.432394 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:03.432395    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d01315b-0855-433a-8e81-337384350872-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:53:04.013282 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:04.013245    2559 generic.go:356] "Generic (PLEG): container finished" podID="1d01315b-0855-433a-8e81-337384350872" containerID="6f3f37d79d3b3cd8548d398653553da10af93eb880c263b552ef88f14a3e463b" exitCode=0
Sep 19 11:53:04.013774 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:04.013292    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"
Sep 19 11:53:04.013774 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:04.013338    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5" event={"ID":"1d01315b-0855-433a-8e81-337384350872","Type":"ContainerDied","Data":"6f3f37d79d3b3cd8548d398653553da10af93eb880c263b552ef88f14a3e463b"}
Sep 19 11:53:04.013774 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:04.013387    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5" event={"ID":"1d01315b-0855-433a-8e81-337384350872","Type":"ContainerDied","Data":"17a15db32077b8edc3001ab628dc3d6d7b41cff8c4b8538ff86295f90eb4b05d"}
Sep 19 11:53:04.013774 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:04.013408    2559 scope.go:117] "RemoveContainer" containerID="6f3f37d79d3b3cd8548d398653553da10af93eb880c263b552ef88f14a3e463b"
Sep 19 11:53:04.023113 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:04.023095    2559 scope.go:117] "RemoveContainer" containerID="6f3f37d79d3b3cd8548d398653553da10af93eb880c263b552ef88f14a3e463b"
Sep 19 11:53:04.023410 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:53:04.023393    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f3f37d79d3b3cd8548d398653553da10af93eb880c263b552ef88f14a3e463b\": container with ID starting with 6f3f37d79d3b3cd8548d398653553da10af93eb880c263b552ef88f14a3e463b not found: ID does not exist" containerID="6f3f37d79d3b3cd8548d398653553da10af93eb880c263b552ef88f14a3e463b"
Sep 19 11:53:04.023462 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:04.023419    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3f37d79d3b3cd8548d398653553da10af93eb880c263b552ef88f14a3e463b"} err="failed to get container status \"6f3f37d79d3b3cd8548d398653553da10af93eb880c263b552ef88f14a3e463b\": rpc error: code = NotFound desc = could not find container \"6f3f37d79d3b3cd8548d398653553da10af93eb880c263b552ef88f14a3e463b\": container with ID starting with 6f3f37d79d3b3cd8548d398653553da10af93eb880c263b552ef88f14a3e463b not found: ID does not exist"
Sep 19 11:53:04.034142 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:04.034117    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"]
Sep 19 11:53:04.044694 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:04.044672    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcfc8fcc-7cbf869477-ddbb5"]
Sep 19 11:53:04.596489 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:04.596453    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05a402a6-e5b8-4fc0-84cc-214f443ffbfd" path="/var/lib/kubelet/pods/05a402a6-e5b8-4fc0-84cc-214f443ffbfd/volumes"
Sep 19 11:53:04.596863 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:04.596850    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d01315b-0855-433a-8e81-337384350872" path="/var/lib/kubelet/pods/1d01315b-0855-433a-8e81-337384350872/volumes"
Sep 19 11:53:08.751768 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:08.751737    2559 scope.go:117] "RemoveContainer" containerID="19a3bc453bf3dfa21d80a0a1b6a6dd4e5c2a494ed94d9b756d2ad101066a4d5c"
Sep 19 11:53:08.761114 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:53:08.761093    2559 scope.go:117] "RemoveContainer" containerID="56a1b52a100459632d881e86f9d8ff8c1f1f9d637db2f368248818efe4dcd588"
Sep 19 11:54:08.789853 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:08.789820    2559 scope.go:117] "RemoveContainer" containerID="315014af181ea358db643119bce40b5f02f6902275f1c74598372394ca7b95a6"
Sep 19 11:54:55.120510 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.120477    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"]
Sep 19 11:54:55.120981 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.120972    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d01315b-0855-433a-8e81-337384350872" containerName="payload-processing"
Sep 19 11:54:55.121028 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.120985    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d01315b-0855-433a-8e81-337384350872" containerName="payload-processing"
Sep 19 11:54:55.121028 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.121016    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05a402a6-e5b8-4fc0-84cc-214f443ffbfd" containerName="revoke-keys"
Sep 19 11:54:55.121028 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.121021    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="05a402a6-e5b8-4fc0-84cc-214f443ffbfd" containerName="revoke-keys"
Sep 19 11:54:55.121126 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.121090    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="05a402a6-e5b8-4fc0-84cc-214f443ffbfd" containerName="revoke-keys"
Sep 19 11:54:55.121126 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.121101    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d01315b-0855-433a-8e81-337384350872" containerName="payload-processing"
Sep 19 11:54:55.124316 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.124301    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"
Sep 19 11:54:55.126815 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.126782    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-3c21ca\""
Sep 19 11:54:55.126926 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.126833    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-3c21ca-dockercfg-52q9c\""
Sep 19 11:54:55.133964 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.133944    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"]
Sep 19 11:54:55.151416 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.151389    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"]
Sep 19 11:54:55.156781 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.156742    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"
Sep 19 11:54:55.158609 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.158584    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"]
Sep 19 11:54:55.218889 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.218859    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kff8k\" (UniqueName: \"kubernetes.io/projected/25f7945e-84c7-4f7c-88ca-ada13c408d5d-kube-api-access-kff8k\") pod \"payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9\" (UID: \"25f7945e-84c7-4f7c-88ca-ada13c408d5d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"
Sep 19 11:54:55.219085 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.218975    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25f7945e-84c7-4f7c-88ca-ada13c408d5d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9\" (UID: \"25f7945e-84c7-4f7c-88ca-ada13c408d5d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"
Sep 19 11:54:55.319913 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.319870    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kff8k\" (UniqueName: \"kubernetes.io/projected/25f7945e-84c7-4f7c-88ca-ada13c408d5d-kube-api-access-kff8k\") pod \"payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9\" (UID: \"25f7945e-84c7-4f7c-88ca-ada13c408d5d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"
Sep 19 11:54:55.320095 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.319966    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25f7945e-84c7-4f7c-88ca-ada13c408d5d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9\" (UID: \"25f7945e-84c7-4f7c-88ca-ada13c408d5d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"
Sep 19 11:54:55.320095 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.319993    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f631873-d0ce-48ec-b864-a34ba3212ae8-plugins-config-volume\") pod \"payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j\" (UID: \"3f631873-d0ce-48ec-b864-a34ba3212ae8\") " pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"
Sep 19 11:54:55.320095 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.320030    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6p4b\" (UniqueName: \"kubernetes.io/projected/3f631873-d0ce-48ec-b864-a34ba3212ae8-kube-api-access-w6p4b\") pod \"payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j\" (UID: \"3f631873-d0ce-48ec-b864-a34ba3212ae8\") " pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"
Sep 19 11:54:55.320667 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.320648    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25f7945e-84c7-4f7c-88ca-ada13c408d5d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9\" (UID: \"25f7945e-84c7-4f7c-88ca-ada13c408d5d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"
Sep 19 11:54:55.327788 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.327770    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kff8k\" (UniqueName: \"kubernetes.io/projected/25f7945e-84c7-4f7c-88ca-ada13c408d5d-kube-api-access-kff8k\") pod \"payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9\" (UID: \"25f7945e-84c7-4f7c-88ca-ada13c408d5d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"
Sep 19 11:54:55.421147 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.421090    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f631873-d0ce-48ec-b864-a34ba3212ae8-plugins-config-volume\") pod \"payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j\" (UID: \"3f631873-d0ce-48ec-b864-a34ba3212ae8\") " pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"
Sep 19 11:54:55.421147 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.421129    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6p4b\" (UniqueName: \"kubernetes.io/projected/3f631873-d0ce-48ec-b864-a34ba3212ae8-kube-api-access-w6p4b\") pod \"payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j\" (UID: \"3f631873-d0ce-48ec-b864-a34ba3212ae8\") " pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"
Sep 19 11:54:55.421778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.421760    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f631873-d0ce-48ec-b864-a34ba3212ae8-plugins-config-volume\") pod \"payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j\" (UID: \"3f631873-d0ce-48ec-b864-a34ba3212ae8\") " pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"
Sep 19 11:54:55.428475 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.428448    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6p4b\" (UniqueName: \"kubernetes.io/projected/3f631873-d0ce-48ec-b864-a34ba3212ae8-kube-api-access-w6p4b\") pod \"payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j\" (UID: \"3f631873-d0ce-48ec-b864-a34ba3212ae8\") " pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"
Sep 19 11:54:55.437357 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.437335    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"
Sep 19 11:54:55.467677 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.467652    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"
Sep 19 11:54:55.713408 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.713387    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"]
Sep 19 11:54:55.714750 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:54:55.714728    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25f7945e_84c7_4f7c_88ca_ada13c408d5d.slice/crio-61ea0426df48b54b5363d58bfddb755e8a88d3e33c17dc239d95ba2493f08b84 WatchSource:0}: Error finding container 61ea0426df48b54b5363d58bfddb755e8a88d3e33c17dc239d95ba2493f08b84: Status 404 returned error can't find the container with id 61ea0426df48b54b5363d58bfddb755e8a88d3e33c17dc239d95ba2493f08b84
Sep 19 11:54:55.745796 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:55.745772    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"]
Sep 19 11:54:55.747272 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:54:55.747249    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f631873_d0ce_48ec_b864_a34ba3212ae8.slice/crio-9a4f6c075e437891fc112838f4b26805cd51342923306362e74c75b8ec9d4fde WatchSource:0}: Error finding container 9a4f6c075e437891fc112838f4b26805cd51342923306362e74c75b8ec9d4fde: Status 404 returned error can't find the container with id 9a4f6c075e437891fc112838f4b26805cd51342923306362e74c75b8ec9d4fde
Sep 19 11:54:56.697529 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:56.697477    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9" event={"ID":"25f7945e-84c7-4f7c-88ca-ada13c408d5d","Type":"ContainerStarted","Data":"c3860bb90deccb33b0ba21ba4e26ba4fb684494c8ce8744afa48dec0a23e8765"}
Sep 19 11:54:56.697983 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:56.697543    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9" event={"ID":"25f7945e-84c7-4f7c-88ca-ada13c408d5d","Type":"ContainerStarted","Data":"61ea0426df48b54b5363d58bfddb755e8a88d3e33c17dc239d95ba2493f08b84"}
Sep 19 11:54:56.697983 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:56.697673    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"
Sep 19 11:54:56.699396 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:56.699371    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j" event={"ID":"3f631873-d0ce-48ec-b864-a34ba3212ae8","Type":"ContainerStarted","Data":"d5ead6798fd9b505f6b9ab2b8898473be102533a23771ca88c9361449bb6a24a"}
Sep 19 11:54:56.699527 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:56.699416    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j" event={"ID":"3f631873-d0ce-48ec-b864-a34ba3212ae8","Type":"ContainerStarted","Data":"9a4f6c075e437891fc112838f4b26805cd51342923306362e74c75b8ec9d4fde"}
Sep 19 11:54:56.699527 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:56.699453    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"
Sep 19 11:54:56.714328 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:56.714285    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9" podStartSLOduration=1.714267771 podStartE2EDuration="1.714267771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:54:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:54:56.711444838 +0000 UTC m=+1248.633757721" watchObservedRunningTime="2026-09-19 11:54:56.714267771 +0000 UTC m=+1248.636580654"
Sep 19 11:54:56.726875 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:54:56.726835    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j" podStartSLOduration=1.726823311 podStartE2EDuration="1.726823311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:54:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:54:56.726168453 +0000 UTC m=+1248.648481334" watchObservedRunningTime="2026-09-19 11:54:56.726823311 +0000 UTC m=+1248.649136192"
Sep 19 11:55:07.707602 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:07.707571    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"
Sep 19 11:55:07.708061 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:07.707653    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"
Sep 19 11:55:08.811424 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:08.811388    2559 scope.go:117] "RemoveContainer" containerID="bdf28b97560c58e5045d38cfcd73d30b40afcf160537407876b5530336948138"
Sep 19 11:55:30.105151 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.105116    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"]
Sep 19 11:55:30.110043 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.110022    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"
Sep 19 11:55:30.112523 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.112502    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-c7b650\""
Sep 19 11:55:30.112630 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.112601    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-c7b650-dockercfg-lsxlv\""
Sep 19 11:55:30.117628 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.117603    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"]
Sep 19 11:55:30.145180 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.145153    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"]
Sep 19 11:55:30.148946 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.148930    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"
Sep 19 11:55:30.160097 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.160072    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"]
Sep 19 11:55:30.262681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.262645    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctjn2\" (UniqueName: \"kubernetes.io/projected/0e70fc04-38fe-4df1-87cb-3de9d6ffa960-kube-api-access-ctjn2\") pod \"payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j\" (UID: \"0e70fc04-38fe-4df1-87cb-3de9d6ffa960\") " pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"
Sep 19 11:55:30.262864 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.262687    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrbl7\" (UniqueName: \"kubernetes.io/projected/d2b9d88d-810a-4606-9fa7-b7fb80307b9f-kube-api-access-lrbl7\") pod \"payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs\" (UID: \"d2b9d88d-810a-4606-9fa7-b7fb80307b9f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"
Sep 19 11:55:30.262864 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.262822    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2b9d88d-810a-4606-9fa7-b7fb80307b9f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs\" (UID: \"d2b9d88d-810a-4606-9fa7-b7fb80307b9f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"
Sep 19 11:55:30.262944 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.262861    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e70fc04-38fe-4df1-87cb-3de9d6ffa960-plugins-config-volume\") pod \"payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j\" (UID: \"0e70fc04-38fe-4df1-87cb-3de9d6ffa960\") " pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"
Sep 19 11:55:30.364020 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.363944    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrbl7\" (UniqueName: \"kubernetes.io/projected/d2b9d88d-810a-4606-9fa7-b7fb80307b9f-kube-api-access-lrbl7\") pod \"payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs\" (UID: \"d2b9d88d-810a-4606-9fa7-b7fb80307b9f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"
Sep 19 11:55:30.364166 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.364023    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2b9d88d-810a-4606-9fa7-b7fb80307b9f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs\" (UID: \"d2b9d88d-810a-4606-9fa7-b7fb80307b9f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"
Sep 19 11:55:30.364166 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.364052    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e70fc04-38fe-4df1-87cb-3de9d6ffa960-plugins-config-volume\") pod \"payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j\" (UID: \"0e70fc04-38fe-4df1-87cb-3de9d6ffa960\") " pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"
Sep 19 11:55:30.364166 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.364085    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctjn2\" (UniqueName: \"kubernetes.io/projected/0e70fc04-38fe-4df1-87cb-3de9d6ffa960-kube-api-access-ctjn2\") pod \"payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j\" (UID: \"0e70fc04-38fe-4df1-87cb-3de9d6ffa960\") " pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"
Sep 19 11:55:30.364811 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.364790    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e70fc04-38fe-4df1-87cb-3de9d6ffa960-plugins-config-volume\") pod \"payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j\" (UID: \"0e70fc04-38fe-4df1-87cb-3de9d6ffa960\") " pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"
Sep 19 11:55:30.364869 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.364796    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2b9d88d-810a-4606-9fa7-b7fb80307b9f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs\" (UID: \"d2b9d88d-810a-4606-9fa7-b7fb80307b9f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"
Sep 19 11:55:30.371846 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.371828    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrbl7\" (UniqueName: \"kubernetes.io/projected/d2b9d88d-810a-4606-9fa7-b7fb80307b9f-kube-api-access-lrbl7\") pod \"payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs\" (UID: \"d2b9d88d-810a-4606-9fa7-b7fb80307b9f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"
Sep 19 11:55:30.371977 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.371957    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctjn2\" (UniqueName: \"kubernetes.io/projected/0e70fc04-38fe-4df1-87cb-3de9d6ffa960-kube-api-access-ctjn2\") pod \"payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j\" (UID: \"0e70fc04-38fe-4df1-87cb-3de9d6ffa960\") " pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"
Sep 19 11:55:30.422393 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.422362    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"
Sep 19 11:55:30.461445 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.461412    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"
Sep 19 11:55:30.694994 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.694970    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"]
Sep 19 11:55:30.696409 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:55:30.696384    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2b9d88d_810a_4606_9fa7_b7fb80307b9f.slice/crio-5fa381340e2a9079f983874268bffb8ee731b197dacca700940d30f7903bb678 WatchSource:0}: Error finding container 5fa381340e2a9079f983874268bffb8ee731b197dacca700940d30f7903bb678: Status 404 returned error can't find the container with id 5fa381340e2a9079f983874268bffb8ee731b197dacca700940d30f7903bb678
Sep 19 11:55:30.746542 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.746390    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"]
Sep 19 11:55:30.747364 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:55:30.747340    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e70fc04_38fe_4df1_87cb_3de9d6ffa960.slice/crio-a35ecff098358df60125ade435eee70a0baad058b25fb65dd82776fa587dfd56 WatchSource:0}: Error finding container a35ecff098358df60125ade435eee70a0baad058b25fb65dd82776fa587dfd56: Status 404 returned error can't find the container with id a35ecff098358df60125ade435eee70a0baad058b25fb65dd82776fa587dfd56
Sep 19 11:55:30.913213 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.913152    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs" event={"ID":"d2b9d88d-810a-4606-9fa7-b7fb80307b9f","Type":"ContainerStarted","Data":"7e4f206e718c72a570c8bdc4839d1dae41e94e0b20011c2d1c1145335893a639"}
Sep 19 11:55:30.913477 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.913247    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs" event={"ID":"d2b9d88d-810a-4606-9fa7-b7fb80307b9f","Type":"ContainerStarted","Data":"5fa381340e2a9079f983874268bffb8ee731b197dacca700940d30f7903bb678"}
Sep 19 11:55:30.913477 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.913394    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"
Sep 19 11:55:30.914969 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.914946    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j" event={"ID":"0e70fc04-38fe-4df1-87cb-3de9d6ffa960","Type":"ContainerStarted","Data":"e9ea4b6dc24c040435488954c8c121888c7ed3c337675b1bce43d07d0bc9d892"}
Sep 19 11:55:30.915101 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.914990    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j" event={"ID":"0e70fc04-38fe-4df1-87cb-3de9d6ffa960","Type":"ContainerStarted","Data":"a35ecff098358df60125ade435eee70a0baad058b25fb65dd82776fa587dfd56"}
Sep 19 11:55:30.915101 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.915082    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"
Sep 19 11:55:30.929151 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.929100    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs" podStartSLOduration=0.929081736 podStartE2EDuration="929.081736ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:55:30.927170317 +0000 UTC m=+1282.849483199" watchObservedRunningTime="2026-09-19 11:55:30.929081736 +0000 UTC m=+1282.851394621"
Sep 19 11:55:30.948307 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:30.948215    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j" podStartSLOduration=0.948182162 podStartE2EDuration="948.182162ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:55:30.944514976 +0000 UTC m=+1282.866827860" watchObservedRunningTime="2026-09-19 11:55:30.948182162 +0000 UTC m=+1282.870495042"
Sep 19 11:55:41.922669 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:41.922639    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"
Sep 19 11:55:41.923151 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:41.922707    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"
Sep 19 11:55:57.982453 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:57.982414    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt"]
Sep 19 11:55:57.986456 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:57.986437    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt"
Sep 19 11:55:57.989390 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:57.989363    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:55:57.992088 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:57.992063    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt"]
Sep 19 11:55:58.039515 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:58.039487    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd68d852-37bb-46a6-ac22-64136198fdb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt\" (UID: \"bd68d852-37bb-46a6-ac22-64136198fdb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt"
Sep 19 11:55:58.140903 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:58.140866    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd68d852-37bb-46a6-ac22-64136198fdb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt\" (UID: \"bd68d852-37bb-46a6-ac22-64136198fdb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt"
Sep 19 11:55:58.141506 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:58.141486    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd68d852-37bb-46a6-ac22-64136198fdb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt\" (UID: \"bd68d852-37bb-46a6-ac22-64136198fdb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt"
Sep 19 11:55:58.301069 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:58.300989    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt"
Sep 19 11:55:58.580001 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:58.579967    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt"]
Sep 19 11:55:58.581667 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:55:58.581644    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd68d852_37bb_46a6_ac22_64136198fdb7.slice/crio-6bec57518aa471867556bafde8e0339e3d4010691be4802b2290747eddc858d1 WatchSource:0}: Error finding container 6bec57518aa471867556bafde8e0339e3d4010691be4802b2290747eddc858d1: Status 404 returned error can't find the container with id 6bec57518aa471867556bafde8e0339e3d4010691be4802b2290747eddc858d1
Sep 19 11:55:59.090974 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:59.090934    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt" event={"ID":"bd68d852-37bb-46a6-ac22-64136198fdb7","Type":"ContainerStarted","Data":"33fc2e1a135ca9dd22347863dcd4388c133472e32bfc98c2734fbfb04de04159"}
Sep 19 11:55:59.091480 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:59.091003    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt" event={"ID":"bd68d852-37bb-46a6-ac22-64136198fdb7","Type":"ContainerStarted","Data":"6bec57518aa471867556bafde8e0339e3d4010691be4802b2290747eddc858d1"}
Sep 19 11:55:59.107089 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:55:59.107040    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt" podStartSLOduration=2.107025287 podStartE2EDuration="2.107025287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:55:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:55:59.104487264 +0000 UTC m=+1311.026800146" watchObservedRunningTime="2026-09-19 11:55:59.107025287 +0000 UTC m=+1311.029338167"
Sep 19 11:56:00.098548 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:00.098514    2559 generic.go:356] "Generic (PLEG): container finished" podID="bd68d852-37bb-46a6-ac22-64136198fdb7" containerID="33fc2e1a135ca9dd22347863dcd4388c133472e32bfc98c2734fbfb04de04159" exitCode=0
Sep 19 11:56:00.098944 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:00.098605    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt" event={"ID":"bd68d852-37bb-46a6-ac22-64136198fdb7","Type":"ContainerDied","Data":"33fc2e1a135ca9dd22347863dcd4388c133472e32bfc98c2734fbfb04de04159"}
Sep 19 11:56:01.245499 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:01.245474    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt"
Sep 19 11:56:01.378580 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:01.378496    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd68d852-37bb-46a6-ac22-64136198fdb7-maas-api-service-ca\") pod \"bd68d852-37bb-46a6-ac22-64136198fdb7\" (UID: \"bd68d852-37bb-46a6-ac22-64136198fdb7\") "
Sep 19 11:56:01.378860 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:01.378834    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd68d852-37bb-46a6-ac22-64136198fdb7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bd68d852-37bb-46a6-ac22-64136198fdb7" (UID: "bd68d852-37bb-46a6-ac22-64136198fdb7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:56:01.479424 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:01.479384    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd68d852-37bb-46a6-ac22-64136198fdb7-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:56:02.111543 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:02.111511    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt"
Sep 19 11:56:02.111726 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:02.111515    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt" event={"ID":"bd68d852-37bb-46a6-ac22-64136198fdb7","Type":"ContainerDied","Data":"6bec57518aa471867556bafde8e0339e3d4010691be4802b2290747eddc858d1"}
Sep 19 11:56:02.111726 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:02.111640    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6bec57518aa471867556bafde8e0339e3d4010691be4802b2290747eddc858d1"
Sep 19 11:56:08.840391 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:08.840358    2559 scope.go:117] "RemoveContainer" containerID="cc964284e45d6d6bd7d95e7b78d44ed2382a80b378e3cb872ebfe2cf89cd03a2"
Sep 19 11:56:13.742703 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:13.742669    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"]
Sep 19 11:56:13.743093 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:13.742910    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j" podUID="3f631873-d0ce-48ec-b864-a34ba3212ae8" containerName="payload-processing" containerID="cri-o://d5ead6798fd9b505f6b9ab2b8898473be102533a23771ca88c9361449bb6a24a" gracePeriod=30
Sep 19 11:56:13.748424 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:13.748393    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"]
Sep 19 11:56:13.748689 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:13.748632    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9" podUID="25f7945e-84c7-4f7c-88ca-ada13c408d5d" containerName="payload-pre-processing" containerID="cri-o://c3860bb90deccb33b0ba21ba4e26ba4fb684494c8ce8744afa48dec0a23e8765" gracePeriod=30
Sep 19 11:56:13.906130 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:13.906045    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt"]
Sep 19 11:56:13.908697 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:13.908668    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3c21ca-4qczt"]
Sep 19 11:56:14.025312 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.025289    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"
Sep 19 11:56:14.041702 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.041677    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"
Sep 19 11:56:14.185843 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.185806    2559 generic.go:356] "Generic (PLEG): container finished" podID="25f7945e-84c7-4f7c-88ca-ada13c408d5d" containerID="c3860bb90deccb33b0ba21ba4e26ba4fb684494c8ce8744afa48dec0a23e8765" exitCode=0
Sep 19 11:56:14.185999 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.185848    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"
Sep 19 11:56:14.185999 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.185885    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9" event={"ID":"25f7945e-84c7-4f7c-88ca-ada13c408d5d","Type":"ContainerDied","Data":"c3860bb90deccb33b0ba21ba4e26ba4fb684494c8ce8744afa48dec0a23e8765"}
Sep 19 11:56:14.185999 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.185943    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9" event={"ID":"25f7945e-84c7-4f7c-88ca-ada13c408d5d","Type":"ContainerDied","Data":"61ea0426df48b54b5363d58bfddb755e8a88d3e33c17dc239d95ba2493f08b84"}
Sep 19 11:56:14.185999 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.185971    2559 scope.go:117] "RemoveContainer" containerID="c3860bb90deccb33b0ba21ba4e26ba4fb684494c8ce8744afa48dec0a23e8765"
Sep 19 11:56:14.187558 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.187533    2559 generic.go:356] "Generic (PLEG): container finished" podID="3f631873-d0ce-48ec-b864-a34ba3212ae8" containerID="d5ead6798fd9b505f6b9ab2b8898473be102533a23771ca88c9361449bb6a24a" exitCode=0
Sep 19 11:56:14.187674 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.187569    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j" event={"ID":"3f631873-d0ce-48ec-b864-a34ba3212ae8","Type":"ContainerDied","Data":"d5ead6798fd9b505f6b9ab2b8898473be102533a23771ca88c9361449bb6a24a"}
Sep 19 11:56:14.187674 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.187592    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"
Sep 19 11:56:14.187674 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.187601    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j" event={"ID":"3f631873-d0ce-48ec-b864-a34ba3212ae8","Type":"ContainerDied","Data":"9a4f6c075e437891fc112838f4b26805cd51342923306362e74c75b8ec9d4fde"}
Sep 19 11:56:14.195300 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.195284    2559 scope.go:117] "RemoveContainer" containerID="c3860bb90deccb33b0ba21ba4e26ba4fb684494c8ce8744afa48dec0a23e8765"
Sep 19 11:56:14.195551 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:56:14.195532    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3860bb90deccb33b0ba21ba4e26ba4fb684494c8ce8744afa48dec0a23e8765\": container with ID starting with c3860bb90deccb33b0ba21ba4e26ba4fb684494c8ce8744afa48dec0a23e8765 not found: ID does not exist" containerID="c3860bb90deccb33b0ba21ba4e26ba4fb684494c8ce8744afa48dec0a23e8765"
Sep 19 11:56:14.195605 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.195559    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3860bb90deccb33b0ba21ba4e26ba4fb684494c8ce8744afa48dec0a23e8765"} err="failed to get container status \"c3860bb90deccb33b0ba21ba4e26ba4fb684494c8ce8744afa48dec0a23e8765\": rpc error: code = NotFound desc = could not find container \"c3860bb90deccb33b0ba21ba4e26ba4fb684494c8ce8744afa48dec0a23e8765\": container with ID starting with c3860bb90deccb33b0ba21ba4e26ba4fb684494c8ce8744afa48dec0a23e8765 not found: ID does not exist"
Sep 19 11:56:14.195605 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.195575    2559 scope.go:117] "RemoveContainer" containerID="d5ead6798fd9b505f6b9ab2b8898473be102533a23771ca88c9361449bb6a24a"
Sep 19 11:56:14.199796 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.199780    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25f7945e-84c7-4f7c-88ca-ada13c408d5d-plugins-config-volume\") pod \"25f7945e-84c7-4f7c-88ca-ada13c408d5d\" (UID: \"25f7945e-84c7-4f7c-88ca-ada13c408d5d\") "
Sep 19 11:56:14.199904 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.199837    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f631873-d0ce-48ec-b864-a34ba3212ae8-plugins-config-volume\") pod \"3f631873-d0ce-48ec-b864-a34ba3212ae8\" (UID: \"3f631873-d0ce-48ec-b864-a34ba3212ae8\") "
Sep 19 11:56:14.199970 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.199956    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6p4b\" (UniqueName: \"kubernetes.io/projected/3f631873-d0ce-48ec-b864-a34ba3212ae8-kube-api-access-w6p4b\") pod \"3f631873-d0ce-48ec-b864-a34ba3212ae8\" (UID: \"3f631873-d0ce-48ec-b864-a34ba3212ae8\") "
Sep 19 11:56:14.200059 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.200026    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kff8k\" (UniqueName: \"kubernetes.io/projected/25f7945e-84c7-4f7c-88ca-ada13c408d5d-kube-api-access-kff8k\") pod \"25f7945e-84c7-4f7c-88ca-ada13c408d5d\" (UID: \"25f7945e-84c7-4f7c-88ca-ada13c408d5d\") "
Sep 19 11:56:14.200230 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.200181    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25f7945e-84c7-4f7c-88ca-ada13c408d5d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "25f7945e-84c7-4f7c-88ca-ada13c408d5d" (UID: "25f7945e-84c7-4f7c-88ca-ada13c408d5d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:56:14.200309 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.200228    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f631873-d0ce-48ec-b864-a34ba3212ae8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3f631873-d0ce-48ec-b864-a34ba3212ae8" (UID: "3f631873-d0ce-48ec-b864-a34ba3212ae8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:56:14.200441 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.200420    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25f7945e-84c7-4f7c-88ca-ada13c408d5d-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:56:14.200519 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.200447    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f631873-d0ce-48ec-b864-a34ba3212ae8-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:56:14.202141 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.202116    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25f7945e-84c7-4f7c-88ca-ada13c408d5d-kube-api-access-kff8k" (OuterVolumeSpecName: "kube-api-access-kff8k") pod "25f7945e-84c7-4f7c-88ca-ada13c408d5d" (UID: "25f7945e-84c7-4f7c-88ca-ada13c408d5d"). InnerVolumeSpecName "kube-api-access-kff8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:56:14.202401 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.202381    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f631873-d0ce-48ec-b864-a34ba3212ae8-kube-api-access-w6p4b" (OuterVolumeSpecName: "kube-api-access-w6p4b") pod "3f631873-d0ce-48ec-b864-a34ba3212ae8" (UID: "3f631873-d0ce-48ec-b864-a34ba3212ae8"). InnerVolumeSpecName "kube-api-access-w6p4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:56:14.204148 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.204130    2559 scope.go:117] "RemoveContainer" containerID="d5ead6798fd9b505f6b9ab2b8898473be102533a23771ca88c9361449bb6a24a"
Sep 19 11:56:14.204471 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:56:14.204443    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5ead6798fd9b505f6b9ab2b8898473be102533a23771ca88c9361449bb6a24a\": container with ID starting with d5ead6798fd9b505f6b9ab2b8898473be102533a23771ca88c9361449bb6a24a not found: ID does not exist" containerID="d5ead6798fd9b505f6b9ab2b8898473be102533a23771ca88c9361449bb6a24a"
Sep 19 11:56:14.204543 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.204469    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5ead6798fd9b505f6b9ab2b8898473be102533a23771ca88c9361449bb6a24a"} err="failed to get container status \"d5ead6798fd9b505f6b9ab2b8898473be102533a23771ca88c9361449bb6a24a\": rpc error: code = NotFound desc = could not find container \"d5ead6798fd9b505f6b9ab2b8898473be102533a23771ca88c9361449bb6a24a\": container with ID starting with d5ead6798fd9b505f6b9ab2b8898473be102533a23771ca88c9361449bb6a24a not found: ID does not exist"
Sep 19 11:56:14.301379 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.301311    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kff8k\" (UniqueName: \"kubernetes.io/projected/25f7945e-84c7-4f7c-88ca-ada13c408d5d-kube-api-access-kff8k\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:56:14.301379 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.301337    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6p4b\" (UniqueName: \"kubernetes.io/projected/3f631873-d0ce-48ec-b864-a34ba3212ae8-kube-api-access-w6p4b\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:56:14.509518 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.509489    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"]
Sep 19 11:56:14.513108 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.513087    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3c21ca-55759886fd-94zb9"]
Sep 19 11:56:14.521532 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.521508    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"]
Sep 19 11:56:14.524933 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.524908    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3c21ca-588f68875c-d7r9j"]
Sep 19 11:56:14.596659 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.596597    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25f7945e-84c7-4f7c-88ca-ada13c408d5d" path="/var/lib/kubelet/pods/25f7945e-84c7-4f7c-88ca-ada13c408d5d/volumes"
Sep 19 11:56:14.597013 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.596999    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f631873-d0ce-48ec-b864-a34ba3212ae8" path="/var/lib/kubelet/pods/3f631873-d0ce-48ec-b864-a34ba3212ae8/volumes"
Sep 19 11:56:14.597362 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:14.597349    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd68d852-37bb-46a6-ac22-64136198fdb7" path="/var/lib/kubelet/pods/bd68d852-37bb-46a6-ac22-64136198fdb7/volumes"
Sep 19 11:56:20.633993 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.633955    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c7b650-cv27x"]
Sep 19 11:56:20.634493 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.634477    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25f7945e-84c7-4f7c-88ca-ada13c408d5d" containerName="payload-pre-processing"
Sep 19 11:56:20.634493 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.634494    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="25f7945e-84c7-4f7c-88ca-ada13c408d5d" containerName="payload-pre-processing"
Sep 19 11:56:20.634596 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.634529    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f631873-d0ce-48ec-b864-a34ba3212ae8" containerName="payload-processing"
Sep 19 11:56:20.634596 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.634539    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f631873-d0ce-48ec-b864-a34ba3212ae8" containerName="payload-processing"
Sep 19 11:56:20.634596 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.634559    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd68d852-37bb-46a6-ac22-64136198fdb7" containerName="revoke-keys"
Sep 19 11:56:20.634596 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.634568    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd68d852-37bb-46a6-ac22-64136198fdb7" containerName="revoke-keys"
Sep 19 11:56:20.634762 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.634640    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f631873-d0ce-48ec-b864-a34ba3212ae8" containerName="payload-processing"
Sep 19 11:56:20.634762 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.634648    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="25f7945e-84c7-4f7c-88ca-ada13c408d5d" containerName="payload-pre-processing"
Sep 19 11:56:20.634762 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.634657    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd68d852-37bb-46a6-ac22-64136198fdb7" containerName="revoke-keys"
Sep 19 11:56:20.639417 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.639402    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c7b650-cv27x"
Sep 19 11:56:20.642378 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.642352    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:56:20.645300 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.645266    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c7b650-cv27x"]
Sep 19 11:56:20.661979 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.661951    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c543c51-fa29-4ca8-a2fe-cd4ab59041b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c7b650-cv27x\" (UID: \"7c543c51-fa29-4ca8-a2fe-cd4ab59041b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c7b650-cv27x"
Sep 19 11:56:20.762684 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.762645    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c543c51-fa29-4ca8-a2fe-cd4ab59041b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c7b650-cv27x\" (UID: \"7c543c51-fa29-4ca8-a2fe-cd4ab59041b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c7b650-cv27x"
Sep 19 11:56:20.763300 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.763281    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c543c51-fa29-4ca8-a2fe-cd4ab59041b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c7b650-cv27x\" (UID: \"7c543c51-fa29-4ca8-a2fe-cd4ab59041b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c7b650-cv27x"
Sep 19 11:56:20.952681 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:20.952652    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c7b650-cv27x"
Sep 19 11:56:21.243056 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:21.243024    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c7b650-cv27x"]
Sep 19 11:56:21.244600 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:56:21.244572    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c543c51_fa29_4ca8_a2fe_cd4ab59041b4.slice/crio-d5e070393936a9271a317696106b41554a486a188226bd5cd890e31f56f0fad0 WatchSource:0}: Error finding container d5e070393936a9271a317696106b41554a486a188226bd5cd890e31f56f0fad0: Status 404 returned error can't find the container with id d5e070393936a9271a317696106b41554a486a188226bd5cd890e31f56f0fad0
Sep 19 11:56:22.247581 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:22.247544    2559 generic.go:356] "Generic (PLEG): container finished" podID="7c543c51-fa29-4ca8-a2fe-cd4ab59041b4" containerID="21e538baa7f0ec033d5fbcd03aaa23f0941c5f06840eae9268f53331eed43851" exitCode=0
Sep 19 11:56:22.247929 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:22.247592    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c7b650-cv27x" event={"ID":"7c543c51-fa29-4ca8-a2fe-cd4ab59041b4","Type":"ContainerDied","Data":"21e538baa7f0ec033d5fbcd03aaa23f0941c5f06840eae9268f53331eed43851"}
Sep 19 11:56:22.247929 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:22.247667    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c7b650-cv27x" event={"ID":"7c543c51-fa29-4ca8-a2fe-cd4ab59041b4","Type":"ContainerStarted","Data":"d5e070393936a9271a317696106b41554a486a188226bd5cd890e31f56f0fad0"}
Sep 19 11:56:23.393247 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:23.393224    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c7b650-cv27x"
Sep 19 11:56:23.489896 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:23.489864    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c543c51-fa29-4ca8-a2fe-cd4ab59041b4-maas-api-service-ca\") pod \"7c543c51-fa29-4ca8-a2fe-cd4ab59041b4\" (UID: \"7c543c51-fa29-4ca8-a2fe-cd4ab59041b4\") "
Sep 19 11:56:23.490252 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:23.490232    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c543c51-fa29-4ca8-a2fe-cd4ab59041b4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7c543c51-fa29-4ca8-a2fe-cd4ab59041b4" (UID: "7c543c51-fa29-4ca8-a2fe-cd4ab59041b4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:56:23.591493 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:23.591413    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c543c51-fa29-4ca8-a2fe-cd4ab59041b4-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:56:24.261108 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:24.261082    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c7b650-cv27x"
Sep 19 11:56:24.261338 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:24.261121    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c7b650-cv27x" event={"ID":"7c543c51-fa29-4ca8-a2fe-cd4ab59041b4","Type":"ContainerDied","Data":"d5e070393936a9271a317696106b41554a486a188226bd5cd890e31f56f0fad0"}
Sep 19 11:56:24.261338 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:24.261171    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5e070393936a9271a317696106b41554a486a188226bd5cd890e31f56f0fad0"
Sep 19 11:56:36.129448 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.129357    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"]
Sep 19 11:56:36.130344 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.130308    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j" podUID="0e70fc04-38fe-4df1-87cb-3de9d6ffa960" containerName="payload-processing" containerID="cri-o://e9ea4b6dc24c040435488954c8c121888c7ed3c337675b1bce43d07d0bc9d892" gracePeriod=30
Sep 19 11:56:36.137541 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.135172    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"]
Sep 19 11:56:36.137541 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.135393    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs" podUID="d2b9d88d-810a-4606-9fa7-b7fb80307b9f" containerName="payload-pre-processing" containerID="cri-o://7e4f206e718c72a570c8bdc4839d1dae41e94e0b20011c2d1c1145335893a639" gracePeriod=30
Sep 19 11:56:36.284260 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.281165    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c7b650-cv27x"]
Sep 19 11:56:36.284260 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.283157    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c7b650-cv27x"]
Sep 19 11:56:36.338243 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.338214    2559 generic.go:356] "Generic (PLEG): container finished" podID="0e70fc04-38fe-4df1-87cb-3de9d6ffa960" containerID="e9ea4b6dc24c040435488954c8c121888c7ed3c337675b1bce43d07d0bc9d892" exitCode=0
Sep 19 11:56:36.338410 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.338299    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j" event={"ID":"0e70fc04-38fe-4df1-87cb-3de9d6ffa960","Type":"ContainerDied","Data":"e9ea4b6dc24c040435488954c8c121888c7ed3c337675b1bce43d07d0bc9d892"}
Sep 19 11:56:36.339972 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.339948    2559 generic.go:356] "Generic (PLEG): container finished" podID="d2b9d88d-810a-4606-9fa7-b7fb80307b9f" containerID="7e4f206e718c72a570c8bdc4839d1dae41e94e0b20011c2d1c1145335893a639" exitCode=0
Sep 19 11:56:36.340093 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.339994    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs" event={"ID":"d2b9d88d-810a-4606-9fa7-b7fb80307b9f","Type":"ContainerDied","Data":"7e4f206e718c72a570c8bdc4839d1dae41e94e0b20011c2d1c1145335893a639"}
Sep 19 11:56:36.435868 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.435843    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"
Sep 19 11:56:36.438987 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.438966    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"
Sep 19 11:56:36.517930 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.517899    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctjn2\" (UniqueName: \"kubernetes.io/projected/0e70fc04-38fe-4df1-87cb-3de9d6ffa960-kube-api-access-ctjn2\") pod \"0e70fc04-38fe-4df1-87cb-3de9d6ffa960\" (UID: \"0e70fc04-38fe-4df1-87cb-3de9d6ffa960\") "
Sep 19 11:56:36.518097 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.518014    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrbl7\" (UniqueName: \"kubernetes.io/projected/d2b9d88d-810a-4606-9fa7-b7fb80307b9f-kube-api-access-lrbl7\") pod \"d2b9d88d-810a-4606-9fa7-b7fb80307b9f\" (UID: \"d2b9d88d-810a-4606-9fa7-b7fb80307b9f\") "
Sep 19 11:56:36.518097 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.518047    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2b9d88d-810a-4606-9fa7-b7fb80307b9f-plugins-config-volume\") pod \"d2b9d88d-810a-4606-9fa7-b7fb80307b9f\" (UID: \"d2b9d88d-810a-4606-9fa7-b7fb80307b9f\") "
Sep 19 11:56:36.518179 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.518112    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e70fc04-38fe-4df1-87cb-3de9d6ffa960-plugins-config-volume\") pod \"0e70fc04-38fe-4df1-87cb-3de9d6ffa960\" (UID: \"0e70fc04-38fe-4df1-87cb-3de9d6ffa960\") "
Sep 19 11:56:36.518534 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.518504    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2b9d88d-810a-4606-9fa7-b7fb80307b9f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d2b9d88d-810a-4606-9fa7-b7fb80307b9f" (UID: "d2b9d88d-810a-4606-9fa7-b7fb80307b9f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:56:36.518671 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.518547    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e70fc04-38fe-4df1-87cb-3de9d6ffa960-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0e70fc04-38fe-4df1-87cb-3de9d6ffa960" (UID: "0e70fc04-38fe-4df1-87cb-3de9d6ffa960"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:56:36.520493 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.520462    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2b9d88d-810a-4606-9fa7-b7fb80307b9f-kube-api-access-lrbl7" (OuterVolumeSpecName: "kube-api-access-lrbl7") pod "d2b9d88d-810a-4606-9fa7-b7fb80307b9f" (UID: "d2b9d88d-810a-4606-9fa7-b7fb80307b9f"). InnerVolumeSpecName "kube-api-access-lrbl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:56:36.520609 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.520503    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e70fc04-38fe-4df1-87cb-3de9d6ffa960-kube-api-access-ctjn2" (OuterVolumeSpecName: "kube-api-access-ctjn2") pod "0e70fc04-38fe-4df1-87cb-3de9d6ffa960" (UID: "0e70fc04-38fe-4df1-87cb-3de9d6ffa960"). InnerVolumeSpecName "kube-api-access-ctjn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:56:36.597679 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.597645    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c543c51-fa29-4ca8-a2fe-cd4ab59041b4" path="/var/lib/kubelet/pods/7c543c51-fa29-4ca8-a2fe-cd4ab59041b4/volumes"
Sep 19 11:56:36.619798 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.619747    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrbl7\" (UniqueName: \"kubernetes.io/projected/d2b9d88d-810a-4606-9fa7-b7fb80307b9f-kube-api-access-lrbl7\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:56:36.619798 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.619783    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2b9d88d-810a-4606-9fa7-b7fb80307b9f-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:56:36.619798 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.619798    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e70fc04-38fe-4df1-87cb-3de9d6ffa960-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:56:36.620039 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:36.619811    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctjn2\" (UniqueName: \"kubernetes.io/projected/0e70fc04-38fe-4df1-87cb-3de9d6ffa960-kube-api-access-ctjn2\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:56:37.348846 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:37.348813    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"
Sep 19 11:56:37.349320 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:37.348858    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j" event={"ID":"0e70fc04-38fe-4df1-87cb-3de9d6ffa960","Type":"ContainerDied","Data":"a35ecff098358df60125ade435eee70a0baad058b25fb65dd82776fa587dfd56"}
Sep 19 11:56:37.349320 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:37.348929    2559 scope.go:117] "RemoveContainer" containerID="e9ea4b6dc24c040435488954c8c121888c7ed3c337675b1bce43d07d0bc9d892"
Sep 19 11:56:37.350355 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:37.350293    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs" event={"ID":"d2b9d88d-810a-4606-9fa7-b7fb80307b9f","Type":"ContainerDied","Data":"5fa381340e2a9079f983874268bffb8ee731b197dacca700940d30f7903bb678"}
Sep 19 11:56:37.350355 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:37.350310    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"
Sep 19 11:56:37.359082 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:37.359064    2559 scope.go:117] "RemoveContainer" containerID="7e4f206e718c72a570c8bdc4839d1dae41e94e0b20011c2d1c1145335893a639"
Sep 19 11:56:37.366914 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:37.366893    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"]
Sep 19 11:56:37.370940 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:37.370920    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c7b650-58b67c6b5d-zf6xs"]
Sep 19 11:56:37.386142 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:37.381751    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"]
Sep 19 11:56:37.391427 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:37.391338    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c7b650-5fb758b44c-kg57j"]
Sep 19 11:56:38.596001 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:38.595968    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e70fc04-38fe-4df1-87cb-3de9d6ffa960" path="/var/lib/kubelet/pods/0e70fc04-38fe-4df1-87cb-3de9d6ffa960/volumes"
Sep 19 11:56:38.596388 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:38.596332    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2b9d88d-810a-4606-9fa7-b7fb80307b9f" path="/var/lib/kubelet/pods/d2b9d88d-810a-4606-9fa7-b7fb80307b9f/volumes"
Sep 19 11:56:42.871068 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:42.871034    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc"]
Sep 19 11:56:42.871526 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:42.871500    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e70fc04-38fe-4df1-87cb-3de9d6ffa960" containerName="payload-processing"
Sep 19 11:56:42.871526 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:42.871511    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e70fc04-38fe-4df1-87cb-3de9d6ffa960" containerName="payload-processing"
Sep 19 11:56:42.871526 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:42.871525    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c543c51-fa29-4ca8-a2fe-cd4ab59041b4" containerName="revoke-keys"
Sep 19 11:56:42.871630 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:42.871530    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c543c51-fa29-4ca8-a2fe-cd4ab59041b4" containerName="revoke-keys"
Sep 19 11:56:42.871630 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:42.871552    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2b9d88d-810a-4606-9fa7-b7fb80307b9f" containerName="payload-pre-processing"
Sep 19 11:56:42.871630 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:42.871558    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2b9d88d-810a-4606-9fa7-b7fb80307b9f" containerName="payload-pre-processing"
Sep 19 11:56:42.871630 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:42.871627    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e70fc04-38fe-4df1-87cb-3de9d6ffa960" containerName="payload-processing"
Sep 19 11:56:42.871754 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:42.871642    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2b9d88d-810a-4606-9fa7-b7fb80307b9f" containerName="payload-pre-processing"
Sep 19 11:56:42.871754 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:42.871649    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c543c51-fa29-4ca8-a2fe-cd4ab59041b4" containerName="revoke-keys"
Sep 19 11:56:42.876232 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:42.876213    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc"
Sep 19 11:56:42.879160 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:42.879133    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:56:42.886872 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:42.885964    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc"]
Sep 19 11:56:42.985859 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:42.985818    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f589869a-ffe1-4a4a-80f2-0b8dee7a730f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc\" (UID: \"f589869a-ffe1-4a4a-80f2-0b8dee7a730f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc"
Sep 19 11:56:43.087090 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:43.087050    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f589869a-ffe1-4a4a-80f2-0b8dee7a730f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc\" (UID: \"f589869a-ffe1-4a4a-80f2-0b8dee7a730f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc"
Sep 19 11:56:43.087710 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:43.087690    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f589869a-ffe1-4a4a-80f2-0b8dee7a730f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc\" (UID: \"f589869a-ffe1-4a4a-80f2-0b8dee7a730f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc"
Sep 19 11:56:43.192224 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:43.192153    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc"
Sep 19 11:56:43.474486 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:43.474456    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc"]
Sep 19 11:56:43.479462 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:56:43.479426    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf589869a_ffe1_4a4a_80f2_0b8dee7a730f.slice/crio-a4020795f0362b72def4e76799b9fac57691acd92fcb445ddd6a0bf6ea3aba6c WatchSource:0}: Error finding container a4020795f0362b72def4e76799b9fac57691acd92fcb445ddd6a0bf6ea3aba6c: Status 404 returned error can't find the container with id a4020795f0362b72def4e76799b9fac57691acd92fcb445ddd6a0bf6ea3aba6c
Sep 19 11:56:44.396038 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:44.395999    2559 generic.go:356] "Generic (PLEG): container finished" podID="f589869a-ffe1-4a4a-80f2-0b8dee7a730f" containerID="2c9d1989201a18ae46f780d1b03ae1064d088ec3676803c9805109d5dec207b5" exitCode=0
Sep 19 11:56:44.396422 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:44.396078    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc" event={"ID":"f589869a-ffe1-4a4a-80f2-0b8dee7a730f","Type":"ContainerDied","Data":"2c9d1989201a18ae46f780d1b03ae1064d088ec3676803c9805109d5dec207b5"}
Sep 19 11:56:44.396422 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:44.396133    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc" event={"ID":"f589869a-ffe1-4a4a-80f2-0b8dee7a730f","Type":"ContainerStarted","Data":"a4020795f0362b72def4e76799b9fac57691acd92fcb445ddd6a0bf6ea3aba6c"}
Sep 19 11:56:45.545818 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:45.545789    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc"
Sep 19 11:56:45.713948 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:45.713918    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f589869a-ffe1-4a4a-80f2-0b8dee7a730f-maas-api-service-ca\") pod \"f589869a-ffe1-4a4a-80f2-0b8dee7a730f\" (UID: \"f589869a-ffe1-4a4a-80f2-0b8dee7a730f\") "
Sep 19 11:56:45.714289 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:45.714261    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f589869a-ffe1-4a4a-80f2-0b8dee7a730f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f589869a-ffe1-4a4a-80f2-0b8dee7a730f" (UID: "f589869a-ffe1-4a4a-80f2-0b8dee7a730f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:56:45.815575 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:45.815546    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f589869a-ffe1-4a4a-80f2-0b8dee7a730f-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:56:46.409582 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:46.409543    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc" event={"ID":"f589869a-ffe1-4a4a-80f2-0b8dee7a730f","Type":"ContainerDied","Data":"a4020795f0362b72def4e76799b9fac57691acd92fcb445ddd6a0bf6ea3aba6c"}
Sep 19 11:56:46.409582 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:46.409570    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc"
Sep 19 11:56:46.409793 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:46.409605    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4020795f0362b72def4e76799b9fac57691acd92fcb445ddd6a0bf6ea3aba6c"
Sep 19 11:56:53.614775 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:53.614746    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"]
Sep 19 11:56:53.615271 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:53.614958    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84" podUID="f7a3758f-2148-48fc-abf8-a6b9ef581726" containerName="payload-pre-processing" containerID="cri-o://fbcc9cffcb1473852194222d44a51e08d62414496b27f7c57e73f064982ba6ce" gracePeriod=30
Sep 19 11:56:53.763088 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:53.763049    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc"]
Sep 19 11:56:53.766042 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:53.766001    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4bb32d-cwbtc"]
Sep 19 11:56:53.923740 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:53.923714    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"
Sep 19 11:56:54.096235 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.096177    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7a3758f-2148-48fc-abf8-a6b9ef581726-plugins-config-volume\") pod \"f7a3758f-2148-48fc-abf8-a6b9ef581726\" (UID: \"f7a3758f-2148-48fc-abf8-a6b9ef581726\") "
Sep 19 11:56:54.096423 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.096301    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79kss\" (UniqueName: \"kubernetes.io/projected/f7a3758f-2148-48fc-abf8-a6b9ef581726-kube-api-access-79kss\") pod \"f7a3758f-2148-48fc-abf8-a6b9ef581726\" (UID: \"f7a3758f-2148-48fc-abf8-a6b9ef581726\") "
Sep 19 11:56:54.096626 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.096597    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7a3758f-2148-48fc-abf8-a6b9ef581726-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7a3758f-2148-48fc-abf8-a6b9ef581726" (UID: "f7a3758f-2148-48fc-abf8-a6b9ef581726"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:56:54.098567 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.098546    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7a3758f-2148-48fc-abf8-a6b9ef581726-kube-api-access-79kss" (OuterVolumeSpecName: "kube-api-access-79kss") pod "f7a3758f-2148-48fc-abf8-a6b9ef581726" (UID: "f7a3758f-2148-48fc-abf8-a6b9ef581726"). InnerVolumeSpecName "kube-api-access-79kss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:56:54.197983 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.197953    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7a3758f-2148-48fc-abf8-a6b9ef581726-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:56:54.197983 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.197978    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79kss\" (UniqueName: \"kubernetes.io/projected/f7a3758f-2148-48fc-abf8-a6b9ef581726-kube-api-access-79kss\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:56:54.457260 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.457159    2559 generic.go:356] "Generic (PLEG): container finished" podID="f7a3758f-2148-48fc-abf8-a6b9ef581726" containerID="fbcc9cffcb1473852194222d44a51e08d62414496b27f7c57e73f064982ba6ce" exitCode=0
Sep 19 11:56:54.457260 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.457244    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84" event={"ID":"f7a3758f-2148-48fc-abf8-a6b9ef581726","Type":"ContainerDied","Data":"fbcc9cffcb1473852194222d44a51e08d62414496b27f7c57e73f064982ba6ce"}
Sep 19 11:56:54.457260 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.457245    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"
Sep 19 11:56:54.457528 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.457316    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84" event={"ID":"f7a3758f-2148-48fc-abf8-a6b9ef581726","Type":"ContainerDied","Data":"9e95deddb89f1a42f27b6c9a8d4adddc537468153d78acc2c14f666dde096e63"}
Sep 19 11:56:54.457528 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.457342    2559 scope.go:117] "RemoveContainer" containerID="fbcc9cffcb1473852194222d44a51e08d62414496b27f7c57e73f064982ba6ce"
Sep 19 11:56:54.469347 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.469327    2559 scope.go:117] "RemoveContainer" containerID="fbcc9cffcb1473852194222d44a51e08d62414496b27f7c57e73f064982ba6ce"
Sep 19 11:56:54.469735 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:56:54.469711    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbcc9cffcb1473852194222d44a51e08d62414496b27f7c57e73f064982ba6ce\": container with ID starting with fbcc9cffcb1473852194222d44a51e08d62414496b27f7c57e73f064982ba6ce not found: ID does not exist" containerID="fbcc9cffcb1473852194222d44a51e08d62414496b27f7c57e73f064982ba6ce"
Sep 19 11:56:54.469818 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.469743    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbcc9cffcb1473852194222d44a51e08d62414496b27f7c57e73f064982ba6ce"} err="failed to get container status \"fbcc9cffcb1473852194222d44a51e08d62414496b27f7c57e73f064982ba6ce\": rpc error: code = NotFound desc = could not find container \"fbcc9cffcb1473852194222d44a51e08d62414496b27f7c57e73f064982ba6ce\": container with ID starting with fbcc9cffcb1473852194222d44a51e08d62414496b27f7c57e73f064982ba6ce not found: ID does not exist"
Sep 19 11:56:54.482461 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.482431    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"]
Sep 19 11:56:54.485944 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.485913    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4bb32d-6df5f8cc66-xkd84"]
Sep 19 11:56:54.596439 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.596403    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f589869a-ffe1-4a4a-80f2-0b8dee7a730f" path="/var/lib/kubelet/pods/f589869a-ffe1-4a4a-80f2-0b8dee7a730f/volumes"
Sep 19 11:56:54.596771 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:56:54.596758    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7a3758f-2148-48fc-abf8-a6b9ef581726" path="/var/lib/kubelet/pods/f7a3758f-2148-48fc-abf8-a6b9ef581726/volumes"
Sep 19 11:57:00.486655 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:00.486618    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8"]
Sep 19 11:57:00.487109 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:00.487096    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f589869a-ffe1-4a4a-80f2-0b8dee7a730f" containerName="revoke-keys"
Sep 19 11:57:00.487159 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:00.487111    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f589869a-ffe1-4a4a-80f2-0b8dee7a730f" containerName="revoke-keys"
Sep 19 11:57:00.487159 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:00.487124    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a3758f-2148-48fc-abf8-a6b9ef581726" containerName="payload-pre-processing"
Sep 19 11:57:00.487159 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:00.487129    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a3758f-2148-48fc-abf8-a6b9ef581726" containerName="payload-pre-processing"
Sep 19 11:57:00.487288 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:00.487213    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f589869a-ffe1-4a4a-80f2-0b8dee7a730f" containerName="revoke-keys"
Sep 19 11:57:00.487288 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:00.487223    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a3758f-2148-48fc-abf8-a6b9ef581726" containerName="payload-pre-processing"
Sep 19 11:57:00.491637 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:00.491617    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8"
Sep 19 11:57:00.494537 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:00.494510    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:57:00.496856 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:00.496832    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8"]
Sep 19 11:57:00.559846 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:00.559816    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d00116b6-cda1-40f1-8a34-545c3661fc93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8\" (UID: \"d00116b6-cda1-40f1-8a34-545c3661fc93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8"
Sep 19 11:57:00.661034 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:00.661003    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d00116b6-cda1-40f1-8a34-545c3661fc93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8\" (UID: \"d00116b6-cda1-40f1-8a34-545c3661fc93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8"
Sep 19 11:57:00.661673 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:00.661653    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d00116b6-cda1-40f1-8a34-545c3661fc93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8\" (UID: \"d00116b6-cda1-40f1-8a34-545c3661fc93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8"
Sep 19 11:57:00.805554 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:00.805468    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8"
Sep 19 11:57:01.076491 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:01.076461    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8"]
Sep 19 11:57:01.077759 ip-10-0-140-136 kubenswrapper[2559]: W0919 11:57:01.077732    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd00116b6_cda1_40f1_8a34_545c3661fc93.slice/crio-6f6f67eaee4022abb9a9baab589eca51a362f56b3fa489590de3e36bc1179434 WatchSource:0}: Error finding container 6f6f67eaee4022abb9a9baab589eca51a362f56b3fa489590de3e36bc1179434: Status 404 returned error can't find the container with id 6f6f67eaee4022abb9a9baab589eca51a362f56b3fa489590de3e36bc1179434
Sep 19 11:57:01.502846 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:01.502796    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8" event={"ID":"d00116b6-cda1-40f1-8a34-545c3661fc93","Type":"ContainerStarted","Data":"fa225fb362a97cdcd902185974d1a396b682378424d2e9f62b5e55c559822045"}
Sep 19 11:57:01.503358 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:01.502857    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8" event={"ID":"d00116b6-cda1-40f1-8a34-545c3661fc93","Type":"ContainerStarted","Data":"6f6f67eaee4022abb9a9baab589eca51a362f56b3fa489590de3e36bc1179434"}
Sep 19 11:57:01.520416 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:01.520370    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8" podStartSLOduration=1.520355924 podStartE2EDuration="1.520355924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:57:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:57:01.518072771 +0000 UTC m=+1373.440385654" watchObservedRunningTime="2026-09-19 11:57:01.520355924 +0000 UTC m=+1373.442668805"
Sep 19 11:57:02.509808 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:02.509772    2559 generic.go:356] "Generic (PLEG): container finished" podID="d00116b6-cda1-40f1-8a34-545c3661fc93" containerID="fa225fb362a97cdcd902185974d1a396b682378424d2e9f62b5e55c559822045" exitCode=0
Sep 19 11:57:02.510289 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:02.509922    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8" event={"ID":"d00116b6-cda1-40f1-8a34-545c3661fc93","Type":"ContainerDied","Data":"fa225fb362a97cdcd902185974d1a396b682378424d2e9f62b5e55c559822045"}
Sep 19 11:57:03.653947 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:03.653920    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8"
Sep 19 11:57:03.691727 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:03.691699    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d00116b6-cda1-40f1-8a34-545c3661fc93-maas-api-service-ca\") pod \"d00116b6-cda1-40f1-8a34-545c3661fc93\" (UID: \"d00116b6-cda1-40f1-8a34-545c3661fc93\") "
Sep 19 11:57:03.692019 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:03.691999    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d00116b6-cda1-40f1-8a34-545c3661fc93-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d00116b6-cda1-40f1-8a34-545c3661fc93" (UID: "d00116b6-cda1-40f1-8a34-545c3661fc93"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:57:03.793125 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:03.793053    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d00116b6-cda1-40f1-8a34-545c3661fc93-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:04.526537 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:04.526510    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8"
Sep 19 11:57:04.526704 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:04.526502    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8" event={"ID":"d00116b6-cda1-40f1-8a34-545c3661fc93","Type":"ContainerDied","Data":"6f6f67eaee4022abb9a9baab589eca51a362f56b3fa489590de3e36bc1179434"}
Sep 19 11:57:04.526704 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:04.526652    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f6f67eaee4022abb9a9baab589eca51a362f56b3fa489590de3e36bc1179434"
Sep 19 11:57:08.027351 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:08.027318    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx"]
Sep 19 11:57:08.027754 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:08.027742    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d00116b6-cda1-40f1-8a34-545c3661fc93" containerName="revoke-keys"
Sep 19 11:57:08.027817 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:08.027755    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d00116b6-cda1-40f1-8a34-545c3661fc93" containerName="revoke-keys"
Sep 19 11:57:08.027853 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:08.027828    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d00116b6-cda1-40f1-8a34-545c3661fc93" containerName="revoke-keys"
Sep 19 11:57:08.030807 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:08.030785    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx"
Sep 19 11:57:08.033990 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:08.033973    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 11:57:08.045208 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:08.045172    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx"]
Sep 19 11:57:08.132927 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:08.132895    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2171004-6f24-4434-a3b1-a71f0414ec7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx\" (UID: \"b2171004-6f24-4434-a3b1-a71f0414ec7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx"
Sep 19 11:57:08.233972 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:08.233939    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2171004-6f24-4434-a3b1-a71f0414ec7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx\" (UID: \"b2171004-6f24-4434-a3b1-a71f0414ec7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx"
Sep 19 11:57:08.234632 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:08.234613    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2171004-6f24-4434-a3b1-a71f0414ec7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx\" (UID: \"b2171004-6f24-4434-a3b1-a71f0414ec7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx"
Sep 19 11:57:08.341493 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:08.341418    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx"
Sep 19 11:57:08.621547 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:08.614855    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx"]
Sep 19 11:57:08.863958 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:08.863877    2559 scope.go:117] "RemoveContainer" containerID="194a4ae258553fccef4476438634b62f37553021452201b0dce85c79b9155485"
Sep 19 11:57:09.557579 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:09.557474    2559 generic.go:356] "Generic (PLEG): container finished" podID="b2171004-6f24-4434-a3b1-a71f0414ec7d" containerID="bfe2255df4600c5d3700a36ed439f460db289f4bc7b879539981bc79e053b0ee" exitCode=0
Sep 19 11:57:09.557956 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:09.557566    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx" event={"ID":"b2171004-6f24-4434-a3b1-a71f0414ec7d","Type":"ContainerDied","Data":"bfe2255df4600c5d3700a36ed439f460db289f4bc7b879539981bc79e053b0ee"}
Sep 19 11:57:09.557956 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:09.557629    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx" event={"ID":"b2171004-6f24-4434-a3b1-a71f0414ec7d","Type":"ContainerStarted","Data":"a4e1a26b6a2fb024f94a7d993018447716b872f48a96f990fd759fdc3a826e73"}
Sep 19 11:57:10.709361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:10.709337    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx"
Sep 19 11:57:10.758778 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:10.758750    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2171004-6f24-4434-a3b1-a71f0414ec7d-maas-api-service-ca\") pod \"b2171004-6f24-4434-a3b1-a71f0414ec7d\" (UID: \"b2171004-6f24-4434-a3b1-a71f0414ec7d\") "
Sep 19 11:57:10.759103 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:10.759083    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2171004-6f24-4434-a3b1-a71f0414ec7d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b2171004-6f24-4434-a3b1-a71f0414ec7d" (UID: "b2171004-6f24-4434-a3b1-a71f0414ec7d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:57:10.860392 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:10.860326    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2171004-6f24-4434-a3b1-a71f0414ec7d-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:11.571671 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:11.571644    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx"
Sep 19 11:57:11.571969 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:11.571678    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx" event={"ID":"b2171004-6f24-4434-a3b1-a71f0414ec7d","Type":"ContainerDied","Data":"a4e1a26b6a2fb024f94a7d993018447716b872f48a96f990fd759fdc3a826e73"}
Sep 19 11:57:11.571969 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:11.571722    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4e1a26b6a2fb024f94a7d993018447716b872f48a96f990fd759fdc3a826e73"
Sep 19 11:57:12.511088 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.511027    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"]
Sep 19 11:57:12.511715 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.511682    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f" podUID="425ddf7d-30d9-46fe-b0c9-0516bf013f8c" containerName="payload-processing" containerID="cri-o://a248aecf2107ced7d6433d96bc1c16a2f8c95722c1feab187b0a44d105af20ec" gracePeriod=30
Sep 19 11:57:12.519067 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.519034    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"]
Sep 19 11:57:12.519506 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.519476    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb" podUID="d8a66d1e-bd04-44e2-b5b5-8357b6734a47" containerName="payload-pre-processing" containerID="cri-o://fb72f1f45d447bdc9fd1741594f7aab1d944557d445a8f510872c99858e50909" gracePeriod=30
Sep 19 11:57:12.665942 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.665909    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8"]
Sep 19 11:57:12.669812 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.669767    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-176c86-98gp8"]
Sep 19 11:57:12.814517 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.814490    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"
Sep 19 11:57:12.816542 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.816522    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"
Sep 19 11:57:12.879512 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.879468    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkkcz\" (UniqueName: \"kubernetes.io/projected/425ddf7d-30d9-46fe-b0c9-0516bf013f8c-kube-api-access-rkkcz\") pod \"425ddf7d-30d9-46fe-b0c9-0516bf013f8c\" (UID: \"425ddf7d-30d9-46fe-b0c9-0516bf013f8c\") "
Sep 19 11:57:12.879966 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.879943    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kb8j\" (UniqueName: \"kubernetes.io/projected/d8a66d1e-bd04-44e2-b5b5-8357b6734a47-kube-api-access-8kb8j\") pod \"d8a66d1e-bd04-44e2-b5b5-8357b6734a47\" (UID: \"d8a66d1e-bd04-44e2-b5b5-8357b6734a47\") "
Sep 19 11:57:12.880218 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.880183    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8a66d1e-bd04-44e2-b5b5-8357b6734a47-plugins-config-volume\") pod \"d8a66d1e-bd04-44e2-b5b5-8357b6734a47\" (UID: \"d8a66d1e-bd04-44e2-b5b5-8357b6734a47\") "
Sep 19 11:57:12.880377 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.880362    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/425ddf7d-30d9-46fe-b0c9-0516bf013f8c-plugins-config-volume\") pod \"425ddf7d-30d9-46fe-b0c9-0516bf013f8c\" (UID: \"425ddf7d-30d9-46fe-b0c9-0516bf013f8c\") "
Sep 19 11:57:12.881116 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.881086    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8a66d1e-bd04-44e2-b5b5-8357b6734a47-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d8a66d1e-bd04-44e2-b5b5-8357b6734a47" (UID: "d8a66d1e-bd04-44e2-b5b5-8357b6734a47"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:57:12.881241 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.881210    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/425ddf7d-30d9-46fe-b0c9-0516bf013f8c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "425ddf7d-30d9-46fe-b0c9-0516bf013f8c" (UID: "425ddf7d-30d9-46fe-b0c9-0516bf013f8c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:57:12.882329 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.882294    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/425ddf7d-30d9-46fe-b0c9-0516bf013f8c-kube-api-access-rkkcz" (OuterVolumeSpecName: "kube-api-access-rkkcz") pod "425ddf7d-30d9-46fe-b0c9-0516bf013f8c" (UID: "425ddf7d-30d9-46fe-b0c9-0516bf013f8c"). InnerVolumeSpecName "kube-api-access-rkkcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:57:12.883103 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.883076    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8a66d1e-bd04-44e2-b5b5-8357b6734a47-kube-api-access-8kb8j" (OuterVolumeSpecName: "kube-api-access-8kb8j") pod "d8a66d1e-bd04-44e2-b5b5-8357b6734a47" (UID: "d8a66d1e-bd04-44e2-b5b5-8357b6734a47"). InnerVolumeSpecName "kube-api-access-8kb8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:57:12.981868 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.981827    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/425ddf7d-30d9-46fe-b0c9-0516bf013f8c-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:12.981868 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.981867    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkkcz\" (UniqueName: \"kubernetes.io/projected/425ddf7d-30d9-46fe-b0c9-0516bf013f8c-kube-api-access-rkkcz\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:12.982127 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.981882    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kb8j\" (UniqueName: \"kubernetes.io/projected/d8a66d1e-bd04-44e2-b5b5-8357b6734a47-kube-api-access-8kb8j\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:12.982127 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:12.981898    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8a66d1e-bd04-44e2-b5b5-8357b6734a47-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:13.586493 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.586454    2559 generic.go:356] "Generic (PLEG): container finished" podID="d8a66d1e-bd04-44e2-b5b5-8357b6734a47" containerID="fb72f1f45d447bdc9fd1741594f7aab1d944557d445a8f510872c99858e50909" exitCode=0
Sep 19 11:57:13.586904 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.586503    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"
Sep 19 11:57:13.586904 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.586544    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb" event={"ID":"d8a66d1e-bd04-44e2-b5b5-8357b6734a47","Type":"ContainerDied","Data":"fb72f1f45d447bdc9fd1741594f7aab1d944557d445a8f510872c99858e50909"}
Sep 19 11:57:13.586904 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.586636    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb" event={"ID":"d8a66d1e-bd04-44e2-b5b5-8357b6734a47","Type":"ContainerDied","Data":"347ec797c949c7a495c8cf247efd0f0912c97b1f333bf0334b097fb9cc35b28e"}
Sep 19 11:57:13.586904 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.586660    2559 scope.go:117] "RemoveContainer" containerID="fb72f1f45d447bdc9fd1741594f7aab1d944557d445a8f510872c99858e50909"
Sep 19 11:57:13.588098 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.588077    2559 generic.go:356] "Generic (PLEG): container finished" podID="425ddf7d-30d9-46fe-b0c9-0516bf013f8c" containerID="a248aecf2107ced7d6433d96bc1c16a2f8c95722c1feab187b0a44d105af20ec" exitCode=0
Sep 19 11:57:13.588153 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.588140    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"
Sep 19 11:57:13.588232 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.588161    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f" event={"ID":"425ddf7d-30d9-46fe-b0c9-0516bf013f8c","Type":"ContainerDied","Data":"a248aecf2107ced7d6433d96bc1c16a2f8c95722c1feab187b0a44d105af20ec"}
Sep 19 11:57:13.588232 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.588225    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f" event={"ID":"425ddf7d-30d9-46fe-b0c9-0516bf013f8c","Type":"ContainerDied","Data":"dfd277a312cedbe57a61b0ed24632135fe25fe4aea955d83ff834644e7384de7"}
Sep 19 11:57:13.597024 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.597007    2559 scope.go:117] "RemoveContainer" containerID="fb72f1f45d447bdc9fd1741594f7aab1d944557d445a8f510872c99858e50909"
Sep 19 11:57:13.597290 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:57:13.597272    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb72f1f45d447bdc9fd1741594f7aab1d944557d445a8f510872c99858e50909\": container with ID starting with fb72f1f45d447bdc9fd1741594f7aab1d944557d445a8f510872c99858e50909 not found: ID does not exist" containerID="fb72f1f45d447bdc9fd1741594f7aab1d944557d445a8f510872c99858e50909"
Sep 19 11:57:13.597365 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.597297    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb72f1f45d447bdc9fd1741594f7aab1d944557d445a8f510872c99858e50909"} err="failed to get container status \"fb72f1f45d447bdc9fd1741594f7aab1d944557d445a8f510872c99858e50909\": rpc error: code = NotFound desc = could not find container \"fb72f1f45d447bdc9fd1741594f7aab1d944557d445a8f510872c99858e50909\": container with ID starting with fb72f1f45d447bdc9fd1741594f7aab1d944557d445a8f510872c99858e50909 not found: ID does not exist"
Sep 19 11:57:13.597365 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.597313    2559 scope.go:117] "RemoveContainer" containerID="a248aecf2107ced7d6433d96bc1c16a2f8c95722c1feab187b0a44d105af20ec"
Sep 19 11:57:13.605548 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.605530    2559 scope.go:117] "RemoveContainer" containerID="a248aecf2107ced7d6433d96bc1c16a2f8c95722c1feab187b0a44d105af20ec"
Sep 19 11:57:13.605808 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:57:13.605790    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a248aecf2107ced7d6433d96bc1c16a2f8c95722c1feab187b0a44d105af20ec\": container with ID starting with a248aecf2107ced7d6433d96bc1c16a2f8c95722c1feab187b0a44d105af20ec not found: ID does not exist" containerID="a248aecf2107ced7d6433d96bc1c16a2f8c95722c1feab187b0a44d105af20ec"
Sep 19 11:57:13.605866 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.605816    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a248aecf2107ced7d6433d96bc1c16a2f8c95722c1feab187b0a44d105af20ec"} err="failed to get container status \"a248aecf2107ced7d6433d96bc1c16a2f8c95722c1feab187b0a44d105af20ec\": rpc error: code = NotFound desc = could not find container \"a248aecf2107ced7d6433d96bc1c16a2f8c95722c1feab187b0a44d105af20ec\": container with ID starting with a248aecf2107ced7d6433d96bc1c16a2f8c95722c1feab187b0a44d105af20ec not found: ID does not exist"
Sep 19 11:57:13.611397 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.611376    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"]
Sep 19 11:57:13.615002 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.614982    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-176c86-6fbfb5d8fd-v8sjb"]
Sep 19 11:57:13.625242 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.625223    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"]
Sep 19 11:57:13.628725 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:13.628705    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-176c86-55d95b777c-tgf8f"]
Sep 19 11:57:14.596098 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:14.596063    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="425ddf7d-30d9-46fe-b0c9-0516bf013f8c" path="/var/lib/kubelet/pods/425ddf7d-30d9-46fe-b0c9-0516bf013f8c/volumes"
Sep 19 11:57:14.596648 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:14.596522    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d00116b6-cda1-40f1-8a34-545c3661fc93" path="/var/lib/kubelet/pods/d00116b6-cda1-40f1-8a34-545c3661fc93/volumes"
Sep 19 11:57:14.596891 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:14.596876    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8a66d1e-bd04-44e2-b5b5-8357b6734a47" path="/var/lib/kubelet/pods/d8a66d1e-bd04-44e2-b5b5-8357b6734a47/volumes"
Sep 19 11:57:23.746412 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:23.746375    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7eb15e-29830305-mfn45"]
Sep 19 11:57:23.749731 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:23.749702    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-7eb15e-29830305-mfn45"]
Sep 19 11:57:23.772745 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:23.772715    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"]
Sep 19 11:57:23.772967 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:23.772936    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w" podUID="35937834-e7f4-48ab-87d2-f9ad269cf8c7" containerName="payload-processing" containerID="cri-o://c155f9440f9b2410bf7527c510a84ed436342e09217ef8a2c70d5c5ff02aca0e" gracePeriod=30
Sep 19 11:57:23.778996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:23.778973    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"]
Sep 19 11:57:23.778996 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:23.782506    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8" podUID="0c17815d-0860-4275-896b-02b6324cb3df" containerName="payload-pre-processing" containerID="cri-o://f55959b71cab5ebd7a0f2b743d3be921a2ba07e0f5b2c3132d935911b654fb5f" gracePeriod=30
Sep 19 11:57:23.999617 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:23.999542    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx"]
Sep 19 11:57:24.013361 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.013315    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-7eb15e-kgkcx"]
Sep 19 11:57:24.081294 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.081255    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"
Sep 19 11:57:24.084825 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.084803    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"
Sep 19 11:57:24.191800 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.191770    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c17815d-0860-4275-896b-02b6324cb3df-plugins-config-volume\") pod \"0c17815d-0860-4275-896b-02b6324cb3df\" (UID: \"0c17815d-0860-4275-896b-02b6324cb3df\") "
Sep 19 11:57:24.191959 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.191821    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fqhp\" (UniqueName: \"kubernetes.io/projected/35937834-e7f4-48ab-87d2-f9ad269cf8c7-kube-api-access-4fqhp\") pod \"35937834-e7f4-48ab-87d2-f9ad269cf8c7\" (UID: \"35937834-e7f4-48ab-87d2-f9ad269cf8c7\") "
Sep 19 11:57:24.191959 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.191915    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqcn9\" (UniqueName: \"kubernetes.io/projected/0c17815d-0860-4275-896b-02b6324cb3df-kube-api-access-xqcn9\") pod \"0c17815d-0860-4275-896b-02b6324cb3df\" (UID: \"0c17815d-0860-4275-896b-02b6324cb3df\") "
Sep 19 11:57:24.192040 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.191964    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35937834-e7f4-48ab-87d2-f9ad269cf8c7-plugins-config-volume\") pod \"35937834-e7f4-48ab-87d2-f9ad269cf8c7\" (UID: \"35937834-e7f4-48ab-87d2-f9ad269cf8c7\") "
Sep 19 11:57:24.192280 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.192205    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c17815d-0860-4275-896b-02b6324cb3df-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0c17815d-0860-4275-896b-02b6324cb3df" (UID: "0c17815d-0860-4275-896b-02b6324cb3df"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:57:24.192403 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.192383    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35937834-e7f4-48ab-87d2-f9ad269cf8c7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "35937834-e7f4-48ab-87d2-f9ad269cf8c7" (UID: "35937834-e7f4-48ab-87d2-f9ad269cf8c7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:57:24.194161 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.194135    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35937834-e7f4-48ab-87d2-f9ad269cf8c7-kube-api-access-4fqhp" (OuterVolumeSpecName: "kube-api-access-4fqhp") pod "35937834-e7f4-48ab-87d2-f9ad269cf8c7" (UID: "35937834-e7f4-48ab-87d2-f9ad269cf8c7"). InnerVolumeSpecName "kube-api-access-4fqhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:57:24.194383 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.194142    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c17815d-0860-4275-896b-02b6324cb3df-kube-api-access-xqcn9" (OuterVolumeSpecName: "kube-api-access-xqcn9") pod "0c17815d-0860-4275-896b-02b6324cb3df" (UID: "0c17815d-0860-4275-896b-02b6324cb3df"). InnerVolumeSpecName "kube-api-access-xqcn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:57:24.293705 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.293639    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35937834-e7f4-48ab-87d2-f9ad269cf8c7-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:24.293705 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.293662    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c17815d-0860-4275-896b-02b6324cb3df-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:24.293705 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.293671    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fqhp\" (UniqueName: \"kubernetes.io/projected/35937834-e7f4-48ab-87d2-f9ad269cf8c7-kube-api-access-4fqhp\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:24.293705 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.293681    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqcn9\" (UniqueName: \"kubernetes.io/projected/0c17815d-0860-4275-896b-02b6324cb3df-kube-api-access-xqcn9\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:24.595183 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.595093    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="945af994-c1f0-4aaa-a01b-75e84b8d0b77" path="/var/lib/kubelet/pods/945af994-c1f0-4aaa-a01b-75e84b8d0b77/volumes"
Sep 19 11:57:24.595493 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.595473    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2171004-6f24-4434-a3b1-a71f0414ec7d" path="/var/lib/kubelet/pods/b2171004-6f24-4434-a3b1-a71f0414ec7d/volumes"
Sep 19 11:57:24.656740 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.656711    2559 generic.go:356] "Generic (PLEG): container finished" podID="35937834-e7f4-48ab-87d2-f9ad269cf8c7" containerID="c155f9440f9b2410bf7527c510a84ed436342e09217ef8a2c70d5c5ff02aca0e" exitCode=0
Sep 19 11:57:24.656869 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.656758    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"
Sep 19 11:57:24.656869 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.656800    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w" event={"ID":"35937834-e7f4-48ab-87d2-f9ad269cf8c7","Type":"ContainerDied","Data":"c155f9440f9b2410bf7527c510a84ed436342e09217ef8a2c70d5c5ff02aca0e"}
Sep 19 11:57:24.656869 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.656858    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w" event={"ID":"35937834-e7f4-48ab-87d2-f9ad269cf8c7","Type":"ContainerDied","Data":"0e85f080559cdd9510ec711af24f728f29073375d24223e47f26e271a0e7ba77"}
Sep 19 11:57:24.657024 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.656882    2559 scope.go:117] "RemoveContainer" containerID="c155f9440f9b2410bf7527c510a84ed436342e09217ef8a2c70d5c5ff02aca0e"
Sep 19 11:57:24.658383 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.658360    2559 generic.go:356] "Generic (PLEG): container finished" podID="0c17815d-0860-4275-896b-02b6324cb3df" containerID="f55959b71cab5ebd7a0f2b743d3be921a2ba07e0f5b2c3132d935911b654fb5f" exitCode=0
Sep 19 11:57:24.658511 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.658410    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"
Sep 19 11:57:24.658511 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.658460    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8" event={"ID":"0c17815d-0860-4275-896b-02b6324cb3df","Type":"ContainerDied","Data":"f55959b71cab5ebd7a0f2b743d3be921a2ba07e0f5b2c3132d935911b654fb5f"}
Sep 19 11:57:24.658620 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.658509    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8" event={"ID":"0c17815d-0860-4275-896b-02b6324cb3df","Type":"ContainerDied","Data":"55e98077750940614054e0d56c7c2b3d75ac3c1de0e787458f1b75e8f85d3ab3"}
Sep 19 11:57:24.669246 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.669229    2559 scope.go:117] "RemoveContainer" containerID="c155f9440f9b2410bf7527c510a84ed436342e09217ef8a2c70d5c5ff02aca0e"
Sep 19 11:57:24.669621 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:57:24.669593    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c155f9440f9b2410bf7527c510a84ed436342e09217ef8a2c70d5c5ff02aca0e\": container with ID starting with c155f9440f9b2410bf7527c510a84ed436342e09217ef8a2c70d5c5ff02aca0e not found: ID does not exist" containerID="c155f9440f9b2410bf7527c510a84ed436342e09217ef8a2c70d5c5ff02aca0e"
Sep 19 11:57:24.669715 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.669623    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c155f9440f9b2410bf7527c510a84ed436342e09217ef8a2c70d5c5ff02aca0e"} err="failed to get container status \"c155f9440f9b2410bf7527c510a84ed436342e09217ef8a2c70d5c5ff02aca0e\": rpc error: code = NotFound desc = could not find container \"c155f9440f9b2410bf7527c510a84ed436342e09217ef8a2c70d5c5ff02aca0e\": container with ID starting with c155f9440f9b2410bf7527c510a84ed436342e09217ef8a2c70d5c5ff02aca0e not found: ID does not exist"
Sep 19 11:57:24.669715 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.669643    2559 scope.go:117] "RemoveContainer" containerID="f55959b71cab5ebd7a0f2b743d3be921a2ba07e0f5b2c3132d935911b654fb5f"
Sep 19 11:57:24.677851 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.677835    2559 scope.go:117] "RemoveContainer" containerID="f55959b71cab5ebd7a0f2b743d3be921a2ba07e0f5b2c3132d935911b654fb5f"
Sep 19 11:57:24.678094 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:57:24.678079    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f55959b71cab5ebd7a0f2b743d3be921a2ba07e0f5b2c3132d935911b654fb5f\": container with ID starting with f55959b71cab5ebd7a0f2b743d3be921a2ba07e0f5b2c3132d935911b654fb5f not found: ID does not exist" containerID="f55959b71cab5ebd7a0f2b743d3be921a2ba07e0f5b2c3132d935911b654fb5f"
Sep 19 11:57:24.678159 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.678100    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55959b71cab5ebd7a0f2b743d3be921a2ba07e0f5b2c3132d935911b654fb5f"} err="failed to get container status \"f55959b71cab5ebd7a0f2b743d3be921a2ba07e0f5b2c3132d935911b654fb5f\": rpc error: code = NotFound desc = could not find container \"f55959b71cab5ebd7a0f2b743d3be921a2ba07e0f5b2c3132d935911b654fb5f\": container with ID starting with f55959b71cab5ebd7a0f2b743d3be921a2ba07e0f5b2c3132d935911b654fb5f not found: ID does not exist"
Sep 19 11:57:24.682752 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.682727    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"]
Sep 19 11:57:24.684829 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.684806    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-7eb15e-86947f9fb6-rfcw8"]
Sep 19 11:57:24.720503 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.720470    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"]
Sep 19 11:57:24.730029 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:24.730009    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-7eb15e-7655745457-wmv9w"]
Sep 19 11:57:26.595001 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:26.594971    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c17815d-0860-4275-896b-02b6324cb3df" path="/var/lib/kubelet/pods/0c17815d-0860-4275-896b-02b6324cb3df/volumes"
Sep 19 11:57:26.595388 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:26.595340    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35937834-e7f4-48ab-87d2-f9ad269cf8c7" path="/var/lib/kubelet/pods/35937834-e7f4-48ab-87d2-f9ad269cf8c7/volumes"
Sep 19 11:57:30.285496 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:30.281970    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"]
Sep 19 11:57:30.285496 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:30.282238    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72" podUID="8109c4b3-2b3e-4f95-bf02-b26cdcda973d" containerName="istio-proxy" containerID="cri-o://dd21ad024f221bcccfc23377f83af8e3a9f77f0d512fcd3366975951f5a11612" gracePeriod=30
Sep 19 11:57:35.540275 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.540252    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:57:35.601909 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.601836    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-envoy\") pod \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") "
Sep 19 11:57:35.601909 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.601869    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-credential-socket\") pod \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") "
Sep 19 11:57:35.601909 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.601889    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-workload-certs\") pod \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") "
Sep 19 11:57:35.601909 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.601905    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-workload-socket\") pod \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") "
Sep 19 11:57:35.602266 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.601945    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gh75b\" (UniqueName: \"kubernetes.io/projected/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-kube-api-access-gh75b\") pod \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") "
Sep 19 11:57:35.602266 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.602020    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istiod-ca-cert\") pod \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") "
Sep 19 11:57:35.602266 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.602047    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-podinfo\") pod \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") "
Sep 19 11:57:35.602266 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.602081    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-data\") pod \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") "
Sep 19 11:57:35.602266 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.602112    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-token\") pod \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\" (UID: \"8109c4b3-2b3e-4f95-bf02-b26cdcda973d\") "
Sep 19 11:57:35.602266 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.602109    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8109c4b3-2b3e-4f95-bf02-b26cdcda973d" (UID: "8109c4b3-2b3e-4f95-bf02-b26cdcda973d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:57:35.602266 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.602157    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8109c4b3-2b3e-4f95-bf02-b26cdcda973d" (UID: "8109c4b3-2b3e-4f95-bf02-b26cdcda973d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:57:35.602266 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.602265    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8109c4b3-2b3e-4f95-bf02-b26cdcda973d" (UID: "8109c4b3-2b3e-4f95-bf02-b26cdcda973d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:57:35.602687 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.602429    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-data" (OuterVolumeSpecName: "istio-data") pod "8109c4b3-2b3e-4f95-bf02-b26cdcda973d" (UID: "8109c4b3-2b3e-4f95-bf02-b26cdcda973d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:57:35.602687 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.602466    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8109c4b3-2b3e-4f95-bf02-b26cdcda973d" (UID: "8109c4b3-2b3e-4f95-bf02-b26cdcda973d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:57:35.602687 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.602487    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-credential-socket\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:35.602687 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.602508    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-workload-certs\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:35.602687 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.602522    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-workload-socket\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:35.604249 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.604173    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8109c4b3-2b3e-4f95-bf02-b26cdcda973d" (UID: "8109c4b3-2b3e-4f95-bf02-b26cdcda973d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:57:35.604406 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.604388    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-token" (OuterVolumeSpecName: "istio-token") pod "8109c4b3-2b3e-4f95-bf02-b26cdcda973d" (UID: "8109c4b3-2b3e-4f95-bf02-b26cdcda973d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:57:35.604776 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.604757    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8109c4b3-2b3e-4f95-bf02-b26cdcda973d" (UID: "8109c4b3-2b3e-4f95-bf02-b26cdcda973d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 11:57:35.604893 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.604781    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-kube-api-access-gh75b" (OuterVolumeSpecName: "kube-api-access-gh75b") pod "8109c4b3-2b3e-4f95-bf02-b26cdcda973d" (UID: "8109c4b3-2b3e-4f95-bf02-b26cdcda973d"). InnerVolumeSpecName "kube-api-access-gh75b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:57:35.703011 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.702982    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gh75b\" (UniqueName: \"kubernetes.io/projected/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-kube-api-access-gh75b\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:35.703011 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.703008    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istiod-ca-cert\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:35.703163 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.703018    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-podinfo\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:35.703163 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.703026    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-data\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:35.703163 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.703034    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-token\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:35.703163 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.703041    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8109c4b3-2b3e-4f95-bf02-b26cdcda973d-istio-envoy\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 11:57:35.723464 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.723439    2559 generic.go:356] "Generic (PLEG): container finished" podID="8109c4b3-2b3e-4f95-bf02-b26cdcda973d" containerID="dd21ad024f221bcccfc23377f83af8e3a9f77f0d512fcd3366975951f5a11612" exitCode=0
Sep 19 11:57:35.723580 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.723489    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"
Sep 19 11:57:35.723580 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.723528    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72" event={"ID":"8109c4b3-2b3e-4f95-bf02-b26cdcda973d","Type":"ContainerDied","Data":"dd21ad024f221bcccfc23377f83af8e3a9f77f0d512fcd3366975951f5a11612"}
Sep 19 11:57:35.723658 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.723583    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72" event={"ID":"8109c4b3-2b3e-4f95-bf02-b26cdcda973d","Type":"ContainerDied","Data":"1d7b7709fd516dd1d1eeb0497813b81684a08b8025367077ec43c6bfe00359ed"}
Sep 19 11:57:35.723658 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.723610    2559 scope.go:117] "RemoveContainer" containerID="dd21ad024f221bcccfc23377f83af8e3a9f77f0d512fcd3366975951f5a11612"
Sep 19 11:57:35.733417 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.733399    2559 scope.go:117] "RemoveContainer" containerID="2f2165ed2580b706bb3e95a59912847fc59788f718ee05061d7c9f9f4eda516e"
Sep 19 11:57:35.741815 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.741790    2559 scope.go:117] "RemoveContainer" containerID="dd21ad024f221bcccfc23377f83af8e3a9f77f0d512fcd3366975951f5a11612"
Sep 19 11:57:35.742572 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:57:35.742546    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd21ad024f221bcccfc23377f83af8e3a9f77f0d512fcd3366975951f5a11612\": container with ID starting with dd21ad024f221bcccfc23377f83af8e3a9f77f0d512fcd3366975951f5a11612 not found: ID does not exist" containerID="dd21ad024f221bcccfc23377f83af8e3a9f77f0d512fcd3366975951f5a11612"
Sep 19 11:57:35.742683 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.742581    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd21ad024f221bcccfc23377f83af8e3a9f77f0d512fcd3366975951f5a11612"} err="failed to get container status \"dd21ad024f221bcccfc23377f83af8e3a9f77f0d512fcd3366975951f5a11612\": rpc error: code = NotFound desc = could not find container \"dd21ad024f221bcccfc23377f83af8e3a9f77f0d512fcd3366975951f5a11612\": container with ID starting with dd21ad024f221bcccfc23377f83af8e3a9f77f0d512fcd3366975951f5a11612 not found: ID does not exist"
Sep 19 11:57:35.742683 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.742602    2559 scope.go:117] "RemoveContainer" containerID="2f2165ed2580b706bb3e95a59912847fc59788f718ee05061d7c9f9f4eda516e"
Sep 19 11:57:35.742944 ip-10-0-140-136 kubenswrapper[2559]: E0919 11:57:35.742925    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f2165ed2580b706bb3e95a59912847fc59788f718ee05061d7c9f9f4eda516e\": container with ID starting with 2f2165ed2580b706bb3e95a59912847fc59788f718ee05061d7c9f9f4eda516e not found: ID does not exist" containerID="2f2165ed2580b706bb3e95a59912847fc59788f718ee05061d7c9f9f4eda516e"
Sep 19 11:57:35.743013 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.742952    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f2165ed2580b706bb3e95a59912847fc59788f718ee05061d7c9f9f4eda516e"} err="failed to get container status \"2f2165ed2580b706bb3e95a59912847fc59788f718ee05061d7c9f9f4eda516e\": rpc error: code = NotFound desc = could not find container \"2f2165ed2580b706bb3e95a59912847fc59788f718ee05061d7c9f9f4eda516e\": container with ID starting with 2f2165ed2580b706bb3e95a59912847fc59788f718ee05061d7c9f9f4eda516e not found: ID does not exist"
Sep 19 11:57:35.744512 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.744493    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"]
Sep 19 11:57:35.750481 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:35.750463    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-7eb15e-openshift-default-ccd5dd84b-wjp72"]
Sep 19 11:57:36.595501 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:57:36.595471    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8109c4b3-2b3e-4f95-bf02-b26cdcda973d" path="/var/lib/kubelet/pods/8109c4b3-2b3e-4f95-bf02-b26cdcda973d/volumes"
Sep 19 11:58:08.915041 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:58:08.914994    2559 scope.go:117] "RemoveContainer" containerID="cb719eeef53d85e80c8668362014a9db59c234916c77e9b45b146449b1b8efa0"
Sep 19 11:58:08.923617 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:58:08.923597    2559 scope.go:117] "RemoveContainer" containerID="ceaa412893cff7be7cffa324fe17af3b5c60ccd091e7060055cbbcfe36ed7832"
Sep 19 11:59:08.966291 ip-10-0-140-136 kubenswrapper[2559]: I0919 11:59:08.966258    2559 scope.go:117] "RemoveContainer" containerID="3051760befd4e6004e68b25c006d61b4d1babb105c87efb4be731da88752c8ec"
Sep 19 12:00:00.131614 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.131526    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-4a8a59-29830320-6zwps"]
Sep 19 12:00:00.132117 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132076    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8109c4b3-2b3e-4f95-bf02-b26cdcda973d" containerName="istio-proxy"
Sep 19 12:00:00.132117 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132092    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8109c4b3-2b3e-4f95-bf02-b26cdcda973d" containerName="istio-proxy"
Sep 19 12:00:00.132117 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132101    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2171004-6f24-4434-a3b1-a71f0414ec7d" containerName="revoke-keys"
Sep 19 12:00:00.132117 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132107    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2171004-6f24-4434-a3b1-a71f0414ec7d" containerName="revoke-keys"
Sep 19 12:00:00.132272 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132121    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35937834-e7f4-48ab-87d2-f9ad269cf8c7" containerName="payload-processing"
Sep 19 12:00:00.132272 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132127    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="35937834-e7f4-48ab-87d2-f9ad269cf8c7" containerName="payload-processing"
Sep 19 12:00:00.132272 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132134    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8a66d1e-bd04-44e2-b5b5-8357b6734a47" containerName="payload-pre-processing"
Sep 19 12:00:00.132272 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132139    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8a66d1e-bd04-44e2-b5b5-8357b6734a47" containerName="payload-pre-processing"
Sep 19 12:00:00.132272 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132146    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c17815d-0860-4275-896b-02b6324cb3df" containerName="payload-pre-processing"
Sep 19 12:00:00.132272 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132152    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c17815d-0860-4275-896b-02b6324cb3df" containerName="payload-pre-processing"
Sep 19 12:00:00.132272 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132170    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="425ddf7d-30d9-46fe-b0c9-0516bf013f8c" containerName="payload-processing"
Sep 19 12:00:00.132272 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132176    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="425ddf7d-30d9-46fe-b0c9-0516bf013f8c" containerName="payload-processing"
Sep 19 12:00:00.132272 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132251    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8109c4b3-2b3e-4f95-bf02-b26cdcda973d" containerName="istio-proxy"
Sep 19 12:00:00.132272 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132259    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="35937834-e7f4-48ab-87d2-f9ad269cf8c7" containerName="payload-processing"
Sep 19 12:00:00.132272 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132264    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c17815d-0860-4275-896b-02b6324cb3df" containerName="payload-pre-processing"
Sep 19 12:00:00.132272 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132269    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8109c4b3-2b3e-4f95-bf02-b26cdcda973d" containerName="istio-proxy"
Sep 19 12:00:00.132272 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132276    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2171004-6f24-4434-a3b1-a71f0414ec7d" containerName="revoke-keys"
Sep 19 12:00:00.132666 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132281    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="425ddf7d-30d9-46fe-b0c9-0516bf013f8c" containerName="payload-processing"
Sep 19 12:00:00.132666 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.132286    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8a66d1e-bd04-44e2-b5b5-8357b6734a47" containerName="payload-pre-processing"
Sep 19 12:00:00.135273 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.135256    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29830320-xj475"]
Sep 19 12:00:00.135388 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.135371    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-4a8a59-29830320-6zwps"
Sep 19 12:00:00.135638 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.135623    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8109c4b3-2b3e-4f95-bf02-b26cdcda973d" containerName="istio-proxy"
Sep 19 12:00:00.135686 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.135641    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8109c4b3-2b3e-4f95-bf02-b26cdcda973d" containerName="istio-proxy"
Sep 19 12:00:00.137379 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.137359    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 12:00:00.138666 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.138648    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830320-xj475"
Sep 19 12:00:00.144684 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.144405    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-4a8a59-29830320-6zwps"]
Sep 19 12:00:00.147383 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.147358    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29830320-xj475"]
Sep 19 12:00:00.149107 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.149091    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-4a8a59-29830320-6zwps"
Sep 19 12:00:00.158142 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.158120    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830320-xj475"
Sep 19 12:00:00.427872 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.427844    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-4a8a59-29830320-6zwps"]
Sep 19 12:00:00.429619 ip-10-0-140-136 kubenswrapper[2559]: W0919 12:00:00.429593    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca3c2846_9a6f_4284_aadc_cdede6f1e706.slice/crio-b2dde1aec10372ba483c10bac16d185318519e5e07c81098b4df6e1159415b7a WatchSource:0}: Error finding container b2dde1aec10372ba483c10bac16d185318519e5e07c81098b4df6e1159415b7a: Status 404 returned error can't find the container with id b2dde1aec10372ba483c10bac16d185318519e5e07c81098b4df6e1159415b7a
Sep 19 12:00:00.437559 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.437537    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29830320-xj475"]
Sep 19 12:00:00.439244 ip-10-0-140-136 kubenswrapper[2559]: W0919 12:00:00.439220    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3990acd7_d918_40ce_9c81_b1b708a07d8d.slice/crio-805048b4b397ad2dbc3b1b5731a38c2860f5ecbdf7c8b55b22856310265e7b54 WatchSource:0}: Error finding container 805048b4b397ad2dbc3b1b5731a38c2860f5ecbdf7c8b55b22856310265e7b54: Status 404 returned error can't find the container with id 805048b4b397ad2dbc3b1b5731a38c2860f5ecbdf7c8b55b22856310265e7b54
Sep 19 12:00:00.522200 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.522157    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830320-xj475" event={"ID":"3990acd7-d918-40ce-9c81-b1b708a07d8d","Type":"ContainerStarted","Data":"805048b4b397ad2dbc3b1b5731a38c2860f5ecbdf7c8b55b22856310265e7b54"}
Sep 19 12:00:00.523454 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:00.523430    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-4a8a59-29830320-6zwps" event={"ID":"ca3c2846-9a6f-4284-aadc-cdede6f1e706","Type":"ContainerStarted","Data":"b2dde1aec10372ba483c10bac16d185318519e5e07c81098b4df6e1159415b7a"}
Sep 19 12:00:01.531331 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:01.531242    2559 generic.go:356] "Generic (PLEG): container finished" podID="3990acd7-d918-40ce-9c81-b1b708a07d8d" containerID="bc7d68c2c94da038d650a0523dc38cea6c207bb3c9d9c2a8ab446cf72f9a8d5e" exitCode=0
Sep 19 12:00:01.531727 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:01.531337    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830320-xj475" event={"ID":"3990acd7-d918-40ce-9c81-b1b708a07d8d","Type":"ContainerDied","Data":"bc7d68c2c94da038d650a0523dc38cea6c207bb3c9d9c2a8ab446cf72f9a8d5e"}
Sep 19 12:00:01.532757 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:01.532737    2559 generic.go:356] "Generic (PLEG): container finished" podID="ca3c2846-9a6f-4284-aadc-cdede6f1e706" containerID="48a899eb1ef58a9d454d0b75bf43751eca7b6d7ae7f2377eba6c484dbedfe61d" exitCode=0
Sep 19 12:00:01.532832 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:01.532812    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-4a8a59-29830320-6zwps" event={"ID":"ca3c2846-9a6f-4284-aadc-cdede6f1e706","Type":"ContainerDied","Data":"48a899eb1ef58a9d454d0b75bf43751eca7b6d7ae7f2377eba6c484dbedfe61d"}
Sep 19 12:00:02.687904 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:02.687883    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-4a8a59-29830320-6zwps"
Sep 19 12:00:02.690978 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:02.690962    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830320-xj475"
Sep 19 12:00:03.545046 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:03.544958    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830320-xj475"
Sep 19 12:00:03.545046 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:03.544987    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830320-xj475" event={"ID":"3990acd7-d918-40ce-9c81-b1b708a07d8d","Type":"ContainerDied","Data":"805048b4b397ad2dbc3b1b5731a38c2860f5ecbdf7c8b55b22856310265e7b54"}
Sep 19 12:00:03.545046 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:03.545036    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="805048b4b397ad2dbc3b1b5731a38c2860f5ecbdf7c8b55b22856310265e7b54"
Sep 19 12:00:03.546382 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:03.546351    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-4a8a59-29830320-6zwps"
Sep 19 12:00:03.546503 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:03.546394    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-4a8a59-29830320-6zwps" event={"ID":"ca3c2846-9a6f-4284-aadc-cdede6f1e706","Type":"ContainerDied","Data":"b2dde1aec10372ba483c10bac16d185318519e5e07c81098b4df6e1159415b7a"}
Sep 19 12:00:03.546503 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:00:03.546439    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2dde1aec10372ba483c10bac16d185318519e5e07c81098b4df6e1159415b7a"
Sep 19 12:01:08.024710 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.024670    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"]
Sep 19 12:01:08.025238 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.025221    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca3c2846-9a6f-4284-aadc-cdede6f1e706" containerName="cleanup"
Sep 19 12:01:08.025291 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.025240    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca3c2846-9a6f-4284-aadc-cdede6f1e706" containerName="cleanup"
Sep 19 12:01:08.025291 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.025257    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3990acd7-d918-40ce-9c81-b1b708a07d8d" containerName="cleanup"
Sep 19 12:01:08.025291 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.025263    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3990acd7-d918-40ce-9c81-b1b708a07d8d" containerName="cleanup"
Sep 19 12:01:08.025386 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.025328    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca3c2846-9a6f-4284-aadc-cdede6f1e706" containerName="cleanup"
Sep 19 12:01:08.025386 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.025335    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3990acd7-d918-40ce-9c81-b1b708a07d8d" containerName="cleanup"
Sep 19 12:01:08.028524 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.028505    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"
Sep 19 12:01:08.031999 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.031968    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-8a618a\""
Sep 19 12:01:08.032110 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.032049    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-8a618a-dockercfg-nzps4\""
Sep 19 12:01:08.042014 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.041992    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"]
Sep 19 12:01:08.060650 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.060621    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"]
Sep 19 12:01:08.065022 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.065002    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"
Sep 19 12:01:08.073887 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.073862    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"]
Sep 19 12:01:08.165686 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.165650    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plnv9\" (UniqueName: \"kubernetes.io/projected/93e189b5-a599-4aaf-958e-243ef7f34813-kube-api-access-plnv9\") pod \"payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz\" (UID: \"93e189b5-a599-4aaf-958e-243ef7f34813\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"
Sep 19 12:01:08.165850 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.165736    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d08a799-35ff-425f-b4bb-a799ba23f18d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg\" (UID: \"8d08a799-35ff-425f-b4bb-a799ba23f18d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"
Sep 19 12:01:08.165850 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.165783    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dmw8\" (UniqueName: \"kubernetes.io/projected/8d08a799-35ff-425f-b4bb-a799ba23f18d-kube-api-access-6dmw8\") pod \"payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg\" (UID: \"8d08a799-35ff-425f-b4bb-a799ba23f18d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"
Sep 19 12:01:08.165850 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.165838    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93e189b5-a599-4aaf-958e-243ef7f34813-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz\" (UID: \"93e189b5-a599-4aaf-958e-243ef7f34813\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"
Sep 19 12:01:08.267337 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.267301    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93e189b5-a599-4aaf-958e-243ef7f34813-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz\" (UID: \"93e189b5-a599-4aaf-958e-243ef7f34813\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"
Sep 19 12:01:08.267546 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.267348    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plnv9\" (UniqueName: \"kubernetes.io/projected/93e189b5-a599-4aaf-958e-243ef7f34813-kube-api-access-plnv9\") pod \"payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz\" (UID: \"93e189b5-a599-4aaf-958e-243ef7f34813\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"
Sep 19 12:01:08.267546 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.267482    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d08a799-35ff-425f-b4bb-a799ba23f18d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg\" (UID: \"8d08a799-35ff-425f-b4bb-a799ba23f18d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"
Sep 19 12:01:08.267665 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.267578    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dmw8\" (UniqueName: \"kubernetes.io/projected/8d08a799-35ff-425f-b4bb-a799ba23f18d-kube-api-access-6dmw8\") pod \"payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg\" (UID: \"8d08a799-35ff-425f-b4bb-a799ba23f18d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"
Sep 19 12:01:08.268041 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.268022    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93e189b5-a599-4aaf-958e-243ef7f34813-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz\" (UID: \"93e189b5-a599-4aaf-958e-243ef7f34813\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"
Sep 19 12:01:08.268261 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.268243    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d08a799-35ff-425f-b4bb-a799ba23f18d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg\" (UID: \"8d08a799-35ff-425f-b4bb-a799ba23f18d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"
Sep 19 12:01:08.275811 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.275750    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plnv9\" (UniqueName: \"kubernetes.io/projected/93e189b5-a599-4aaf-958e-243ef7f34813-kube-api-access-plnv9\") pod \"payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz\" (UID: \"93e189b5-a599-4aaf-958e-243ef7f34813\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"
Sep 19 12:01:08.275944 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.275923    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dmw8\" (UniqueName: \"kubernetes.io/projected/8d08a799-35ff-425f-b4bb-a799ba23f18d-kube-api-access-6dmw8\") pod \"payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg\" (UID: \"8d08a799-35ff-425f-b4bb-a799ba23f18d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"
Sep 19 12:01:08.339359 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.339323    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"
Sep 19 12:01:08.376568 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.376516    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"
Sep 19 12:01:08.614668 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.614643    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"]
Sep 19 12:01:08.616160 ip-10-0-140-136 kubenswrapper[2559]: W0919 12:01:08.616138    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d08a799_35ff_425f_b4bb_a799ba23f18d.slice/crio-5bc2ee694c3b06b7e5be6342ebbb55cfef56d1df0f3cca253a315754c2e6dfe2 WatchSource:0}: Error finding container 5bc2ee694c3b06b7e5be6342ebbb55cfef56d1df0f3cca253a315754c2e6dfe2: Status 404 returned error can't find the container with id 5bc2ee694c3b06b7e5be6342ebbb55cfef56d1df0f3cca253a315754c2e6dfe2
Sep 19 12:01:08.666786 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.666762    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"]
Sep 19 12:01:08.667804 ip-10-0-140-136 kubenswrapper[2559]: W0919 12:01:08.667780    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e189b5_a599_4aaf_958e_243ef7f34813.slice/crio-847e21156188f8130c4603f5d6de4fed41ef3747a4867e5e95d786b77a7d8d9f WatchSource:0}: Error finding container 847e21156188f8130c4603f5d6de4fed41ef3747a4867e5e95d786b77a7d8d9f: Status 404 returned error can't find the container with id 847e21156188f8130c4603f5d6de4fed41ef3747a4867e5e95d786b77a7d8d9f
Sep 19 12:01:08.909531 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.909364    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg" event={"ID":"8d08a799-35ff-425f-b4bb-a799ba23f18d","Type":"ContainerStarted","Data":"7859cd8d6da70de78bf5f3de0a8ca6eb4450fa9cbde452ab9f961eb514d6748d"}
Sep 19 12:01:08.909531 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.909432    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg" event={"ID":"8d08a799-35ff-425f-b4bb-a799ba23f18d","Type":"ContainerStarted","Data":"5bc2ee694c3b06b7e5be6342ebbb55cfef56d1df0f3cca253a315754c2e6dfe2"}
Sep 19 12:01:08.909765 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.909635    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"
Sep 19 12:01:08.911543 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.911514    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz" event={"ID":"93e189b5-a599-4aaf-958e-243ef7f34813","Type":"ContainerStarted","Data":"8f4445e6c2c54bec8837d6518ebdca9326a21a27b513360d7b9b9f0eb591a0c5"}
Sep 19 12:01:08.911708 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.911563    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz" event={"ID":"93e189b5-a599-4aaf-958e-243ef7f34813","Type":"ContainerStarted","Data":"847e21156188f8130c4603f5d6de4fed41ef3747a4867e5e95d786b77a7d8d9f"}
Sep 19 12:01:08.911708 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.911631    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"
Sep 19 12:01:08.929295 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.929251    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg" podStartSLOduration=0.929237727 podStartE2EDuration="929.237727ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:01:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:01:08.927012597 +0000 UTC m=+1620.849325477" watchObservedRunningTime="2026-09-19 12:01:08.929237727 +0000 UTC m=+1620.851550608"
Sep 19 12:01:08.944469 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:08.944425    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz" podStartSLOduration=0.944415394 podStartE2EDuration="944.415394ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:01:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:01:08.942739049 +0000 UTC m=+1620.865051930" watchObservedRunningTime="2026-09-19 12:01:08.944415394 +0000 UTC m=+1620.866728275"
Sep 19 12:01:19.922023 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:19.921993    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"
Sep 19 12:01:19.922472 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:19.922063    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"
Sep 19 12:01:37.727030 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.726991    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"]
Sep 19 12:01:37.730784 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.730765    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"
Sep 19 12:01:37.733140 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.733118    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-df64a0\""
Sep 19 12:01:37.733262 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.733245    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-df64a0-dockercfg-s8dbp\""
Sep 19 12:01:37.740026 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.740003    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"]
Sep 19 12:01:37.766712 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.766684    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"]
Sep 19 12:01:37.770151 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.770132    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"
Sep 19 12:01:37.780537 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.780516    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"]
Sep 19 12:01:37.850492 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.850454    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4mjm\" (UniqueName: \"kubernetes.io/projected/ecfefade-7e7d-466e-bcad-5968111e13d6-kube-api-access-p4mjm\") pod \"payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g\" (UID: \"ecfefade-7e7d-466e-bcad-5968111e13d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"
Sep 19 12:01:37.850706 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.850521    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea5fef3d-37b1-423d-bea4-847142a0310c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k\" (UID: \"ea5fef3d-37b1-423d-bea4-847142a0310c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"
Sep 19 12:01:37.850706 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.850572    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fm8v\" (UniqueName: \"kubernetes.io/projected/ea5fef3d-37b1-423d-bea4-847142a0310c-kube-api-access-8fm8v\") pod \"payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k\" (UID: \"ea5fef3d-37b1-423d-bea4-847142a0310c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"
Sep 19 12:01:37.850706 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.850600    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecfefade-7e7d-466e-bcad-5968111e13d6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g\" (UID: \"ecfefade-7e7d-466e-bcad-5968111e13d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"
Sep 19 12:01:37.951640 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.951600    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4mjm\" (UniqueName: \"kubernetes.io/projected/ecfefade-7e7d-466e-bcad-5968111e13d6-kube-api-access-p4mjm\") pod \"payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g\" (UID: \"ecfefade-7e7d-466e-bcad-5968111e13d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"
Sep 19 12:01:37.951826 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.951654    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea5fef3d-37b1-423d-bea4-847142a0310c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k\" (UID: \"ea5fef3d-37b1-423d-bea4-847142a0310c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"
Sep 19 12:01:37.951826 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.951699    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fm8v\" (UniqueName: \"kubernetes.io/projected/ea5fef3d-37b1-423d-bea4-847142a0310c-kube-api-access-8fm8v\") pod \"payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k\" (UID: \"ea5fef3d-37b1-423d-bea4-847142a0310c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"
Sep 19 12:01:37.951826 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.951728    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecfefade-7e7d-466e-bcad-5968111e13d6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g\" (UID: \"ecfefade-7e7d-466e-bcad-5968111e13d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"
Sep 19 12:01:37.952405 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.952387    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea5fef3d-37b1-423d-bea4-847142a0310c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k\" (UID: \"ea5fef3d-37b1-423d-bea4-847142a0310c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"
Sep 19 12:01:37.952460 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.952435    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecfefade-7e7d-466e-bcad-5968111e13d6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g\" (UID: \"ecfefade-7e7d-466e-bcad-5968111e13d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"
Sep 19 12:01:37.959372 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.959348    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4mjm\" (UniqueName: \"kubernetes.io/projected/ecfefade-7e7d-466e-bcad-5968111e13d6-kube-api-access-p4mjm\") pod \"payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g\" (UID: \"ecfefade-7e7d-466e-bcad-5968111e13d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"
Sep 19 12:01:37.959461 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:37.959436    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fm8v\" (UniqueName: \"kubernetes.io/projected/ea5fef3d-37b1-423d-bea4-847142a0310c-kube-api-access-8fm8v\") pod \"payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k\" (UID: \"ea5fef3d-37b1-423d-bea4-847142a0310c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"
Sep 19 12:01:38.042498 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:38.042414    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"
Sep 19 12:01:38.082207 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:38.082014    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"
Sep 19 12:01:38.316538 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:38.316509    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"]
Sep 19 12:01:38.318075 ip-10-0-140-136 kubenswrapper[2559]: W0919 12:01:38.318050    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecfefade_7e7d_466e_bcad_5968111e13d6.slice/crio-5d4c26bbfd7860ff728a94a8161de5c4a6bfc448cf9dd47355628c71439fae93 WatchSource:0}: Error finding container 5d4c26bbfd7860ff728a94a8161de5c4a6bfc448cf9dd47355628c71439fae93: Status 404 returned error can't find the container with id 5d4c26bbfd7860ff728a94a8161de5c4a6bfc448cf9dd47355628c71439fae93
Sep 19 12:01:38.361508 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:38.361473    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"]
Sep 19 12:01:38.374480 ip-10-0-140-136 kubenswrapper[2559]: W0919 12:01:38.374454    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea5fef3d_37b1_423d_bea4_847142a0310c.slice/crio-7cbfa55ea5b487321b9d9a42aa2c071b5dcc6c7e798a4e3ffb151092926699a9 WatchSource:0}: Error finding container 7cbfa55ea5b487321b9d9a42aa2c071b5dcc6c7e798a4e3ffb151092926699a9: Status 404 returned error can't find the container with id 7cbfa55ea5b487321b9d9a42aa2c071b5dcc6c7e798a4e3ffb151092926699a9
Sep 19 12:01:39.090869 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:39.090829    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k" event={"ID":"ea5fef3d-37b1-423d-bea4-847142a0310c","Type":"ContainerStarted","Data":"1b10cb1c9615f090494769cf8951f855c7c1bbed47180a9e24460d2c80e89d8a"}
Sep 19 12:01:39.091326 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:39.090931    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k" event={"ID":"ea5fef3d-37b1-423d-bea4-847142a0310c","Type":"ContainerStarted","Data":"7cbfa55ea5b487321b9d9a42aa2c071b5dcc6c7e798a4e3ffb151092926699a9"}
Sep 19 12:01:39.091326 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:39.090963    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"
Sep 19 12:01:39.092496 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:39.092464    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g" event={"ID":"ecfefade-7e7d-466e-bcad-5968111e13d6","Type":"ContainerStarted","Data":"7280a1e38cab6a5e6a53c82049c7c007dcaa2952a953556530f1bb06da8896b7"}
Sep 19 12:01:39.092631 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:39.092519    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g" event={"ID":"ecfefade-7e7d-466e-bcad-5968111e13d6","Type":"ContainerStarted","Data":"5d4c26bbfd7860ff728a94a8161de5c4a6bfc448cf9dd47355628c71439fae93"}
Sep 19 12:01:39.092631 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:39.092551    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"
Sep 19 12:01:39.105175 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:39.105125    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k" podStartSLOduration=2.105106941 podStartE2EDuration="2.105106941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:01:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:01:39.104466763 +0000 UTC m=+1651.026779645" watchObservedRunningTime="2026-09-19 12:01:39.105106941 +0000 UTC m=+1651.027419826"
Sep 19 12:01:39.120621 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:39.120569    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g" podStartSLOduration=2.120553568 podStartE2EDuration="2.120553568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:01:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:01:39.118091969 +0000 UTC m=+1651.040404851" watchObservedRunningTime="2026-09-19 12:01:39.120553568 +0000 UTC m=+1651.042866445"
Sep 19 12:01:50.100557 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:50.100525    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"
Sep 19 12:01:50.101028 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:01:50.100599    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"
Sep 19 12:02:05.219930 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:05.219892    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-cf2vb_a6292959-58d7-4e70-9bd2-ddc765c83de0/payload-processing/0.log"
Sep 19 12:02:05.339018 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:05.338988    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k_ea5fef3d-37b1-423d-bea4-847142a0310c/payload-processing/0.log"
Sep 19 12:02:09.007786 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:09.007759    2559 scope.go:117] "RemoveContainer" containerID="33fc2e1a135ca9dd22347863dcd4388c133472e32bfc98c2734fbfb04de04159"
Sep 19 12:02:48.578842 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:48.578813    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz_93e189b5-a599-4aaf-958e-243ef7f34813/payload-processing/0.log"
Sep 19 12:02:48.694631 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:48.694604    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-cf2vb_a6292959-58d7-4e70-9bd2-ddc765c83de0/payload-processing/0.log"
Sep 19 12:02:48.813075 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:48.813049    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k_ea5fef3d-37b1-423d-bea4-847142a0310c/payload-processing/0.log"
Sep 19 12:02:48.936477 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:48.936458    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k_ea5fef3d-37b1-423d-bea4-847142a0310c/payload-processing/0.log"
Sep 19 12:02:56.405892 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:56.405852    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"]
Sep 19 12:02:56.409802 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:56.409781    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"
Sep 19 12:02:56.412042 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:56.412022    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-78dcba-dockercfg-5fxwd\""
Sep 19 12:02:56.412157 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:56.412028    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-78dcba\""
Sep 19 12:02:56.419944 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:56.419874    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"]
Sep 19 12:02:56.507694 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:56.507658    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42j4w\" (UniqueName: \"kubernetes.io/projected/abaf1841-ba03-4966-ba07-2c197b416a10-kube-api-access-42j4w\") pod \"payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl\" (UID: \"abaf1841-ba03-4966-ba07-2c197b416a10\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"
Sep 19 12:02:56.507694 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:56.507691    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abaf1841-ba03-4966-ba07-2c197b416a10-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl\" (UID: \"abaf1841-ba03-4966-ba07-2c197b416a10\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"
Sep 19 12:02:56.608573 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:56.608536    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42j4w\" (UniqueName: \"kubernetes.io/projected/abaf1841-ba03-4966-ba07-2c197b416a10-kube-api-access-42j4w\") pod \"payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl\" (UID: \"abaf1841-ba03-4966-ba07-2c197b416a10\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"
Sep 19 12:02:56.608765 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:56.608579    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abaf1841-ba03-4966-ba07-2c197b416a10-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl\" (UID: \"abaf1841-ba03-4966-ba07-2c197b416a10\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"
Sep 19 12:02:56.609291 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:56.609262    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abaf1841-ba03-4966-ba07-2c197b416a10-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl\" (UID: \"abaf1841-ba03-4966-ba07-2c197b416a10\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"
Sep 19 12:02:56.615881 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:56.615858    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42j4w\" (UniqueName: \"kubernetes.io/projected/abaf1841-ba03-4966-ba07-2c197b416a10-kube-api-access-42j4w\") pod \"payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl\" (UID: \"abaf1841-ba03-4966-ba07-2c197b416a10\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"
Sep 19 12:02:56.722495 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:56.722464    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"
Sep 19 12:02:56.990529 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:56.990505    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"]
Sep 19 12:02:56.993269 ip-10-0-140-136 kubenswrapper[2559]: W0919 12:02:56.993238    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabaf1841_ba03_4966_ba07_2c197b416a10.slice/crio-78e4b533d73592194c9c1b781e90208bff88b86128761370753e792ef0f75124 WatchSource:0}: Error finding container 78e4b533d73592194c9c1b781e90208bff88b86128761370753e792ef0f75124: Status 404 returned error can't find the container with id 78e4b533d73592194c9c1b781e90208bff88b86128761370753e792ef0f75124
Sep 19 12:02:57.549886 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:57.549851    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl" event={"ID":"abaf1841-ba03-4966-ba07-2c197b416a10","Type":"ContainerStarted","Data":"37cd0b82b6a173a8117e9154c7bce7baa77cd3b43aa113dfcc52dfe407f95505"}
Sep 19 12:02:57.550359 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:57.549908    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl" event={"ID":"abaf1841-ba03-4966-ba07-2c197b416a10","Type":"ContainerStarted","Data":"78e4b533d73592194c9c1b781e90208bff88b86128761370753e792ef0f75124"}
Sep 19 12:02:57.550359 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:57.549930    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"
Sep 19 12:02:57.565901 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:02:57.565853    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl" podStartSLOduration=1.565837384 podStartE2EDuration="1.565837384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:02:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:02:57.56306935 +0000 UTC m=+1729.485382231" watchObservedRunningTime="2026-09-19 12:02:57.565837384 +0000 UTC m=+1729.488150265"
Sep 19 12:03:08.558118 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:08.558085    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"
Sep 19 12:03:09.029517 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:09.029481    2559 scope.go:117] "RemoveContainer" containerID="21e538baa7f0ec033d5fbcd03aaa23f0941c5f06840eae9268f53331eed43851"
Sep 19 12:03:09.038322 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:09.038299    2559 scope.go:117] "RemoveContainer" containerID="bfe2255df4600c5d3700a36ed439f460db289f4bc7b879539981bc79e053b0ee"
Sep 19 12:03:09.046487 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:09.046467    2559 scope.go:117] "RemoveContainer" containerID="fa225fb362a97cdcd902185974d1a396b682378424d2e9f62b5e55c559822045"
Sep 19 12:03:09.054327 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:09.054311    2559 scope.go:117] "RemoveContainer" containerID="2c9d1989201a18ae46f780d1b03ae1064d088ec3676803c9805109d5dec207b5"
Sep 19 12:03:18.261641 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:18.261605    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9"]
Sep 19 12:03:18.265260 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:18.265243    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9"
Sep 19 12:03:18.267381 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:18.267363    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 12:03:18.272230 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:18.272209    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9"]
Sep 19 12:03:18.316824 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:18.316787    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9f924b7-573b-4469-85ad-eeb05ff86739-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9\" (UID: \"b9f924b7-573b-4469-85ad-eeb05ff86739\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9"
Sep 19 12:03:18.417875 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:18.417841    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9f924b7-573b-4469-85ad-eeb05ff86739-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9\" (UID: \"b9f924b7-573b-4469-85ad-eeb05ff86739\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9"
Sep 19 12:03:18.418504 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:18.418487    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9f924b7-573b-4469-85ad-eeb05ff86739-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9\" (UID: \"b9f924b7-573b-4469-85ad-eeb05ff86739\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9"
Sep 19 12:03:18.577016 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:18.576934    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9"
Sep 19 12:03:18.848218 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:18.848168    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9"]
Sep 19 12:03:18.849914 ip-10-0-140-136 kubenswrapper[2559]: W0919 12:03:18.849891    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f924b7_573b_4469_85ad_eeb05ff86739.slice/crio-ad71f06638bafe3954d10eff61d5365f53439cb3afc98d0288fa69132273011d WatchSource:0}: Error finding container ad71f06638bafe3954d10eff61d5365f53439cb3afc98d0288fa69132273011d: Status 404 returned error can't find the container with id ad71f06638bafe3954d10eff61d5365f53439cb3afc98d0288fa69132273011d
Sep 19 12:03:19.688571 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:19.688535    2559 generic.go:356] "Generic (PLEG): container finished" podID="b9f924b7-573b-4469-85ad-eeb05ff86739" containerID="28fab2d9d61b04b44a0ea80580ddef297e8117e5b223d99eb4ed58a5a91eedb2" exitCode=0
Sep 19 12:03:19.688924 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:19.688593    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9" event={"ID":"b9f924b7-573b-4469-85ad-eeb05ff86739","Type":"ContainerDied","Data":"28fab2d9d61b04b44a0ea80580ddef297e8117e5b223d99eb4ed58a5a91eedb2"}
Sep 19 12:03:19.688924 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:19.688643    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9" event={"ID":"b9f924b7-573b-4469-85ad-eeb05ff86739","Type":"ContainerStarted","Data":"ad71f06638bafe3954d10eff61d5365f53439cb3afc98d0288fa69132273011d"}
Sep 19 12:03:20.836328 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:20.836300    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9"
Sep 19 12:03:20.843866 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:20.843846    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9f924b7-573b-4469-85ad-eeb05ff86739-maas-api-service-ca\") pod \"b9f924b7-573b-4469-85ad-eeb05ff86739\" (UID: \"b9f924b7-573b-4469-85ad-eeb05ff86739\") "
Sep 19 12:03:20.844181 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:20.844161    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9f924b7-573b-4469-85ad-eeb05ff86739-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b9f924b7-573b-4469-85ad-eeb05ff86739" (UID: "b9f924b7-573b-4469-85ad-eeb05ff86739"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:03:20.944857 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:20.944778    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9f924b7-573b-4469-85ad-eeb05ff86739-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:03:21.700415 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:21.700383    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9"
Sep 19 12:03:21.700624 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:21.700421    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9" event={"ID":"b9f924b7-573b-4469-85ad-eeb05ff86739","Type":"ContainerDied","Data":"ad71f06638bafe3954d10eff61d5365f53439cb3afc98d0288fa69132273011d"}
Sep 19 12:03:21.700624 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:21.700471    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad71f06638bafe3954d10eff61d5365f53439cb3afc98d0288fa69132273011d"
Sep 19 12:03:24.564891 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:24.564832    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"]
Sep 19 12:03:24.566393 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:24.566359    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl" podUID="abaf1841-ba03-4966-ba07-2c197b416a10" containerName="payload-pre-processing" containerID="cri-o://37cd0b82b6a173a8117e9154c7bce7baa77cd3b43aa113dfcc52dfe407f95505" gracePeriod=30
Sep 19 12:03:24.725899 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:24.725825    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9"]
Sep 19 12:03:24.727807 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:24.727778    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-78dcba-nvqr9"]
Sep 19 12:03:24.732252 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:24.732172    2559 generic.go:356] "Generic (PLEG): container finished" podID="abaf1841-ba03-4966-ba07-2c197b416a10" containerID="37cd0b82b6a173a8117e9154c7bce7baa77cd3b43aa113dfcc52dfe407f95505" exitCode=0
Sep 19 12:03:24.732364 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:24.732246    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl" event={"ID":"abaf1841-ba03-4966-ba07-2c197b416a10","Type":"ContainerDied","Data":"37cd0b82b6a173a8117e9154c7bce7baa77cd3b43aa113dfcc52dfe407f95505"}
Sep 19 12:03:24.846952 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:24.846928    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"
Sep 19 12:03:24.884354 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:24.884317    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42j4w\" (UniqueName: \"kubernetes.io/projected/abaf1841-ba03-4966-ba07-2c197b416a10-kube-api-access-42j4w\") pod \"abaf1841-ba03-4966-ba07-2c197b416a10\" (UID: \"abaf1841-ba03-4966-ba07-2c197b416a10\") "
Sep 19 12:03:24.884547 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:24.884368    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abaf1841-ba03-4966-ba07-2c197b416a10-plugins-config-volume\") pod \"abaf1841-ba03-4966-ba07-2c197b416a10\" (UID: \"abaf1841-ba03-4966-ba07-2c197b416a10\") "
Sep 19 12:03:24.884850 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:24.884816    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abaf1841-ba03-4966-ba07-2c197b416a10-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "abaf1841-ba03-4966-ba07-2c197b416a10" (UID: "abaf1841-ba03-4966-ba07-2c197b416a10"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:03:24.886569 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:24.886539    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abaf1841-ba03-4966-ba07-2c197b416a10-kube-api-access-42j4w" (OuterVolumeSpecName: "kube-api-access-42j4w") pod "abaf1841-ba03-4966-ba07-2c197b416a10" (UID: "abaf1841-ba03-4966-ba07-2c197b416a10"). InnerVolumeSpecName "kube-api-access-42j4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:03:24.985569 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:24.985539    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42j4w\" (UniqueName: \"kubernetes.io/projected/abaf1841-ba03-4966-ba07-2c197b416a10-kube-api-access-42j4w\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:03:24.985569 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:24.985565    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abaf1841-ba03-4966-ba07-2c197b416a10-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:03:25.739284 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:25.739253    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"
Sep 19 12:03:25.739284 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:25.739283    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl" event={"ID":"abaf1841-ba03-4966-ba07-2c197b416a10","Type":"ContainerDied","Data":"78e4b533d73592194c9c1b781e90208bff88b86128761370753e792ef0f75124"}
Sep 19 12:03:25.739775 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:25.739344    2559 scope.go:117] "RemoveContainer" containerID="37cd0b82b6a173a8117e9154c7bce7baa77cd3b43aa113dfcc52dfe407f95505"
Sep 19 12:03:25.760130 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:25.760102    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"]
Sep 19 12:03:25.767518 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:25.767495    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-78dcba-7d77d94c98-t9xdl"]
Sep 19 12:03:26.595990 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:26.595956    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abaf1841-ba03-4966-ba07-2c197b416a10" path="/var/lib/kubelet/pods/abaf1841-ba03-4966-ba07-2c197b416a10/volumes"
Sep 19 12:03:26.596350 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:26.596336    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9f924b7-573b-4469-85ad-eeb05ff86739" path="/var/lib/kubelet/pods/b9f924b7-573b-4469-85ad-eeb05ff86739/volumes"
Sep 19 12:03:34.803139 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:34.803108    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj"]
Sep 19 12:03:34.803725 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:34.803573    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abaf1841-ba03-4966-ba07-2c197b416a10" containerName="payload-pre-processing"
Sep 19 12:03:34.803725 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:34.803585    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="abaf1841-ba03-4966-ba07-2c197b416a10" containerName="payload-pre-processing"
Sep 19 12:03:34.803725 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:34.803600    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9f924b7-573b-4469-85ad-eeb05ff86739" containerName="revoke-keys"
Sep 19 12:03:34.803725 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:34.803605    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9f924b7-573b-4469-85ad-eeb05ff86739" containerName="revoke-keys"
Sep 19 12:03:34.803725 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:34.803678    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="abaf1841-ba03-4966-ba07-2c197b416a10" containerName="payload-pre-processing"
Sep 19 12:03:34.803725 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:34.803688    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9f924b7-573b-4469-85ad-eeb05ff86739" containerName="revoke-keys"
Sep 19 12:03:34.807580 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:34.807565    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj"
Sep 19 12:03:34.810411 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:34.810385    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 12:03:34.812915 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:34.812891    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj"]
Sep 19 12:03:34.877663 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:34.877623    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e04c136a-f051-471d-8079-9e81612e158c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj\" (UID: \"e04c136a-f051-471d-8079-9e81612e158c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj"
Sep 19 12:03:34.979109 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:34.979076    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e04c136a-f051-471d-8079-9e81612e158c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj\" (UID: \"e04c136a-f051-471d-8079-9e81612e158c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj"
Sep 19 12:03:34.979700 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:34.979682    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e04c136a-f051-471d-8079-9e81612e158c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj\" (UID: \"e04c136a-f051-471d-8079-9e81612e158c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj"
Sep 19 12:03:35.119637 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:35.119560    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj"
Sep 19 12:03:35.390151 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:35.390125    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj"]
Sep 19 12:03:35.391030 ip-10-0-140-136 kubenswrapper[2559]: W0919 12:03:35.391004    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode04c136a_f051_471d_8079_9e81612e158c.slice/crio-dcbec2561d5490a3fdb01a238f2533696879e9faeb63b9e6b2729b9a52e01de2 WatchSource:0}: Error finding container dcbec2561d5490a3fdb01a238f2533696879e9faeb63b9e6b2729b9a52e01de2: Status 404 returned error can't find the container with id dcbec2561d5490a3fdb01a238f2533696879e9faeb63b9e6b2729b9a52e01de2
Sep 19 12:03:35.799820 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:35.799787    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj" event={"ID":"e04c136a-f051-471d-8079-9e81612e158c","Type":"ContainerStarted","Data":"fd016c5993002d029ead51f0fbd7f2afd53c4da29f62aaac6ff01d983e73e1ac"}
Sep 19 12:03:35.799985 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:35.799850    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj" event={"ID":"e04c136a-f051-471d-8079-9e81612e158c","Type":"ContainerStarted","Data":"dcbec2561d5490a3fdb01a238f2533696879e9faeb63b9e6b2729b9a52e01de2"}
Sep 19 12:03:35.814173 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:35.814124    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj" podStartSLOduration=1.814108721 podStartE2EDuration="1.814108721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:03:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:03:35.813892437 +0000 UTC m=+1767.736205312" watchObservedRunningTime="2026-09-19 12:03:35.814108721 +0000 UTC m=+1767.736421602"
Sep 19 12:03:36.806657 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:36.806620    2559 generic.go:356] "Generic (PLEG): container finished" podID="e04c136a-f051-471d-8079-9e81612e158c" containerID="fd016c5993002d029ead51f0fbd7f2afd53c4da29f62aaac6ff01d983e73e1ac" exitCode=0
Sep 19 12:03:36.806857 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:36.806665    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj" event={"ID":"e04c136a-f051-471d-8079-9e81612e158c","Type":"ContainerDied","Data":"fd016c5993002d029ead51f0fbd7f2afd53c4da29f62aaac6ff01d983e73e1ac"}
Sep 19 12:03:37.948747 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:37.948725    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj"
Sep 19 12:03:38.010716 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:38.010686    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e04c136a-f051-471d-8079-9e81612e158c-maas-api-service-ca\") pod \"e04c136a-f051-471d-8079-9e81612e158c\" (UID: \"e04c136a-f051-471d-8079-9e81612e158c\") "
Sep 19 12:03:38.010993 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:38.010971    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e04c136a-f051-471d-8079-9e81612e158c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e04c136a-f051-471d-8079-9e81612e158c" (UID: "e04c136a-f051-471d-8079-9e81612e158c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:03:38.112290 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:38.112212    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e04c136a-f051-471d-8079-9e81612e158c-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:03:38.818921 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:38.818881    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj" event={"ID":"e04c136a-f051-471d-8079-9e81612e158c","Type":"ContainerDied","Data":"dcbec2561d5490a3fdb01a238f2533696879e9faeb63b9e6b2729b9a52e01de2"}
Sep 19 12:03:38.819087 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:38.818933    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dcbec2561d5490a3fdb01a238f2533696879e9faeb63b9e6b2729b9a52e01de2"
Sep 19 12:03:38.819087 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:38.818889    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj"
Sep 19 12:03:50.311319 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.311277    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"]
Sep 19 12:03:50.311794 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.311543    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz" podUID="93e189b5-a599-4aaf-958e-243ef7f34813" containerName="payload-processing" containerID="cri-o://8f4445e6c2c54bec8837d6518ebdca9326a21a27b513360d7b9b9f0eb591a0c5" gracePeriod=30
Sep 19 12:03:50.318091 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.318061    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"]
Sep 19 12:03:50.318333 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.318301    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg" podUID="8d08a799-35ff-425f-b4bb-a799ba23f18d" containerName="payload-pre-processing" containerID="cri-o://7859cd8d6da70de78bf5f3de0a8ca6eb4450fa9cbde452ab9f961eb514d6748d" gracePeriod=30
Sep 19 12:03:50.465283 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.465235    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj"]
Sep 19 12:03:50.467703 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.467672    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a618a-jj2xj"]
Sep 19 12:03:50.591646 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.591618    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"
Sep 19 12:03:50.595136 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.595109    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e04c136a-f051-471d-8079-9e81612e158c" path="/var/lib/kubelet/pods/e04c136a-f051-471d-8079-9e81612e158c/volumes"
Sep 19 12:03:50.618595 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.618566    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"
Sep 19 12:03:50.729644 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.729606    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plnv9\" (UniqueName: \"kubernetes.io/projected/93e189b5-a599-4aaf-958e-243ef7f34813-kube-api-access-plnv9\") pod \"93e189b5-a599-4aaf-958e-243ef7f34813\" (UID: \"93e189b5-a599-4aaf-958e-243ef7f34813\") "
Sep 19 12:03:50.729644 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.729641    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dmw8\" (UniqueName: \"kubernetes.io/projected/8d08a799-35ff-425f-b4bb-a799ba23f18d-kube-api-access-6dmw8\") pod \"8d08a799-35ff-425f-b4bb-a799ba23f18d\" (UID: \"8d08a799-35ff-425f-b4bb-a799ba23f18d\") "
Sep 19 12:03:50.729871 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.729668    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93e189b5-a599-4aaf-958e-243ef7f34813-plugins-config-volume\") pod \"93e189b5-a599-4aaf-958e-243ef7f34813\" (UID: \"93e189b5-a599-4aaf-958e-243ef7f34813\") "
Sep 19 12:03:50.729871 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.729808    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d08a799-35ff-425f-b4bb-a799ba23f18d-plugins-config-volume\") pod \"8d08a799-35ff-425f-b4bb-a799ba23f18d\" (UID: \"8d08a799-35ff-425f-b4bb-a799ba23f18d\") "
Sep 19 12:03:50.730102 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.730082    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93e189b5-a599-4aaf-958e-243ef7f34813-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "93e189b5-a599-4aaf-958e-243ef7f34813" (UID: "93e189b5-a599-4aaf-958e-243ef7f34813"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:03:50.730170 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.730148    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d08a799-35ff-425f-b4bb-a799ba23f18d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8d08a799-35ff-425f-b4bb-a799ba23f18d" (UID: "8d08a799-35ff-425f-b4bb-a799ba23f18d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:03:50.730326 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.730305    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93e189b5-a599-4aaf-958e-243ef7f34813-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:03:50.730326 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.730326    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d08a799-35ff-425f-b4bb-a799ba23f18d-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:03:50.731978 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.731957    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d08a799-35ff-425f-b4bb-a799ba23f18d-kube-api-access-6dmw8" (OuterVolumeSpecName: "kube-api-access-6dmw8") pod "8d08a799-35ff-425f-b4bb-a799ba23f18d" (UID: "8d08a799-35ff-425f-b4bb-a799ba23f18d"). InnerVolumeSpecName "kube-api-access-6dmw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:03:50.732051 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.731977    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93e189b5-a599-4aaf-958e-243ef7f34813-kube-api-access-plnv9" (OuterVolumeSpecName: "kube-api-access-plnv9") pod "93e189b5-a599-4aaf-958e-243ef7f34813" (UID: "93e189b5-a599-4aaf-958e-243ef7f34813"). InnerVolumeSpecName "kube-api-access-plnv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:03:50.831112 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.831033    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plnv9\" (UniqueName: \"kubernetes.io/projected/93e189b5-a599-4aaf-958e-243ef7f34813-kube-api-access-plnv9\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:03:50.831112 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.831058    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dmw8\" (UniqueName: \"kubernetes.io/projected/8d08a799-35ff-425f-b4bb-a799ba23f18d-kube-api-access-6dmw8\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:03:50.887620 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.887584    2559 generic.go:356] "Generic (PLEG): container finished" podID="8d08a799-35ff-425f-b4bb-a799ba23f18d" containerID="7859cd8d6da70de78bf5f3de0a8ca6eb4450fa9cbde452ab9f961eb514d6748d" exitCode=0
Sep 19 12:03:50.887776 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.887635    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"
Sep 19 12:03:50.887776 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.887668    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg" event={"ID":"8d08a799-35ff-425f-b4bb-a799ba23f18d","Type":"ContainerDied","Data":"7859cd8d6da70de78bf5f3de0a8ca6eb4450fa9cbde452ab9f961eb514d6748d"}
Sep 19 12:03:50.887776 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.887738    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg" event={"ID":"8d08a799-35ff-425f-b4bb-a799ba23f18d","Type":"ContainerDied","Data":"5bc2ee694c3b06b7e5be6342ebbb55cfef56d1df0f3cca253a315754c2e6dfe2"}
Sep 19 12:03:50.887776 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.887760    2559 scope.go:117] "RemoveContainer" containerID="7859cd8d6da70de78bf5f3de0a8ca6eb4450fa9cbde452ab9f961eb514d6748d"
Sep 19 12:03:50.889117 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.889094    2559 generic.go:356] "Generic (PLEG): container finished" podID="93e189b5-a599-4aaf-958e-243ef7f34813" containerID="8f4445e6c2c54bec8837d6518ebdca9326a21a27b513360d7b9b9f0eb591a0c5" exitCode=0
Sep 19 12:03:50.889223 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.889145    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"
Sep 19 12:03:50.889223 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.889160    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz" event={"ID":"93e189b5-a599-4aaf-958e-243ef7f34813","Type":"ContainerDied","Data":"8f4445e6c2c54bec8837d6518ebdca9326a21a27b513360d7b9b9f0eb591a0c5"}
Sep 19 12:03:50.889365 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.889228    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz" event={"ID":"93e189b5-a599-4aaf-958e-243ef7f34813","Type":"ContainerDied","Data":"847e21156188f8130c4603f5d6de4fed41ef3747a4867e5e95d786b77a7d8d9f"}
Sep 19 12:03:50.898675 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.898654    2559 scope.go:117] "RemoveContainer" containerID="7859cd8d6da70de78bf5f3de0a8ca6eb4450fa9cbde452ab9f961eb514d6748d"
Sep 19 12:03:50.898997 ip-10-0-140-136 kubenswrapper[2559]: E0919 12:03:50.898968    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7859cd8d6da70de78bf5f3de0a8ca6eb4450fa9cbde452ab9f961eb514d6748d\": container with ID starting with 7859cd8d6da70de78bf5f3de0a8ca6eb4450fa9cbde452ab9f961eb514d6748d not found: ID does not exist" containerID="7859cd8d6da70de78bf5f3de0a8ca6eb4450fa9cbde452ab9f961eb514d6748d"
Sep 19 12:03:50.899106 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.899007    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7859cd8d6da70de78bf5f3de0a8ca6eb4450fa9cbde452ab9f961eb514d6748d"} err="failed to get container status \"7859cd8d6da70de78bf5f3de0a8ca6eb4450fa9cbde452ab9f961eb514d6748d\": rpc error: code = NotFound desc = could not find container \"7859cd8d6da70de78bf5f3de0a8ca6eb4450fa9cbde452ab9f961eb514d6748d\": container with ID starting with 7859cd8d6da70de78bf5f3de0a8ca6eb4450fa9cbde452ab9f961eb514d6748d not found: ID does not exist"
Sep 19 12:03:50.899106 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.899030    2559 scope.go:117] "RemoveContainer" containerID="8f4445e6c2c54bec8837d6518ebdca9326a21a27b513360d7b9b9f0eb591a0c5"
Sep 19 12:03:50.911741 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.911724    2559 scope.go:117] "RemoveContainer" containerID="8f4445e6c2c54bec8837d6518ebdca9326a21a27b513360d7b9b9f0eb591a0c5"
Sep 19 12:03:50.912082 ip-10-0-140-136 kubenswrapper[2559]: E0919 12:03:50.912059    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f4445e6c2c54bec8837d6518ebdca9326a21a27b513360d7b9b9f0eb591a0c5\": container with ID starting with 8f4445e6c2c54bec8837d6518ebdca9326a21a27b513360d7b9b9f0eb591a0c5 not found: ID does not exist" containerID="8f4445e6c2c54bec8837d6518ebdca9326a21a27b513360d7b9b9f0eb591a0c5"
Sep 19 12:03:50.912082 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.912074    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"]
Sep 19 12:03:50.912207 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.912090    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f4445e6c2c54bec8837d6518ebdca9326a21a27b513360d7b9b9f0eb591a0c5"} err="failed to get container status \"8f4445e6c2c54bec8837d6518ebdca9326a21a27b513360d7b9b9f0eb591a0c5\": rpc error: code = NotFound desc = could not find container \"8f4445e6c2c54bec8837d6518ebdca9326a21a27b513360d7b9b9f0eb591a0c5\": container with ID starting with 8f4445e6c2c54bec8837d6518ebdca9326a21a27b513360d7b9b9f0eb591a0c5 not found: ID does not exist"
Sep 19 12:03:50.914456 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.914434    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a618a-6f46cd894-w6nmg"]
Sep 19 12:03:50.923117 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.923074    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"]
Sep 19 12:03:50.929774 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:50.929751    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8a618a-8457d49ddc-d2nqz"]
Sep 19 12:03:52.595528 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:52.595495    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d08a799-35ff-425f-b4bb-a799ba23f18d" path="/var/lib/kubelet/pods/8d08a799-35ff-425f-b4bb-a799ba23f18d/volumes"
Sep 19 12:03:52.595891 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:03:52.595851    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93e189b5-a599-4aaf-958e-243ef7f34813" path="/var/lib/kubelet/pods/93e189b5-a599-4aaf-958e-243ef7f34813/volumes"
Sep 19 12:04:38.926773 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:38.926735    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx"]
Sep 19 12:04:38.927373 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:38.927229    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e04c136a-f051-471d-8079-9e81612e158c" containerName="revoke-keys"
Sep 19 12:04:38.927373 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:38.927242    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e04c136a-f051-471d-8079-9e81612e158c" containerName="revoke-keys"
Sep 19 12:04:38.927373 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:38.927272    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d08a799-35ff-425f-b4bb-a799ba23f18d" containerName="payload-pre-processing"
Sep 19 12:04:38.927373 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:38.927278    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d08a799-35ff-425f-b4bb-a799ba23f18d" containerName="payload-pre-processing"
Sep 19 12:04:38.927373 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:38.927290    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93e189b5-a599-4aaf-958e-243ef7f34813" containerName="payload-processing"
Sep 19 12:04:38.927373 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:38.927296    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="93e189b5-a599-4aaf-958e-243ef7f34813" containerName="payload-processing"
Sep 19 12:04:38.927373 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:38.927369    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d08a799-35ff-425f-b4bb-a799ba23f18d" containerName="payload-pre-processing"
Sep 19 12:04:38.927373 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:38.927381    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="93e189b5-a599-4aaf-958e-243ef7f34813" containerName="payload-processing"
Sep 19 12:04:38.927693 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:38.927388    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e04c136a-f051-471d-8079-9e81612e158c" containerName="revoke-keys"
Sep 19 12:04:38.930593 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:38.930577    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx"
Sep 19 12:04:38.932849 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:38.932825    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 12:04:38.937667 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:38.937625    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx"]
Sep 19 12:04:39.078581 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:39.078535    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e123a4ac-0ccf-42c4-82c3-9272a7f5b67d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx\" (UID: \"e123a4ac-0ccf-42c4-82c3-9272a7f5b67d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx"
Sep 19 12:04:39.179392 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:39.179370    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e123a4ac-0ccf-42c4-82c3-9272a7f5b67d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx\" (UID: \"e123a4ac-0ccf-42c4-82c3-9272a7f5b67d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx"
Sep 19 12:04:39.179963 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:39.179944    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e123a4ac-0ccf-42c4-82c3-9272a7f5b67d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx\" (UID: \"e123a4ac-0ccf-42c4-82c3-9272a7f5b67d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx"
Sep 19 12:04:39.243296 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:39.243272    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx"
Sep 19 12:04:39.513591 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:39.513568    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx"]
Sep 19 12:04:39.515411 ip-10-0-140-136 kubenswrapper[2559]: W0919 12:04:39.515385    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode123a4ac_0ccf_42c4_82c3_9272a7f5b67d.slice/crio-39696a860ed2295baeb248985644717eee9586dc714ea78a00d175bb5dc7416f WatchSource:0}: Error finding container 39696a860ed2295baeb248985644717eee9586dc714ea78a00d175bb5dc7416f: Status 404 returned error can't find the container with id 39696a860ed2295baeb248985644717eee9586dc714ea78a00d175bb5dc7416f
Sep 19 12:04:40.158921 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:40.158888    2559 generic.go:356] "Generic (PLEG): container finished" podID="e123a4ac-0ccf-42c4-82c3-9272a7f5b67d" containerID="78cb2a7c3981d2aa7328aa8aa3c570b800f19887b9eec82389c16853eff98ab7" exitCode=0
Sep 19 12:04:40.159282 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:40.158979    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx" event={"ID":"e123a4ac-0ccf-42c4-82c3-9272a7f5b67d","Type":"ContainerDied","Data":"78cb2a7c3981d2aa7328aa8aa3c570b800f19887b9eec82389c16853eff98ab7"}
Sep 19 12:04:40.159282 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:40.159036    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx" event={"ID":"e123a4ac-0ccf-42c4-82c3-9272a7f5b67d","Type":"ContainerStarted","Data":"39696a860ed2295baeb248985644717eee9586dc714ea78a00d175bb5dc7416f"}
Sep 19 12:04:41.305555 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:41.305526    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx"
Sep 19 12:04:41.400547 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:41.400513    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e123a4ac-0ccf-42c4-82c3-9272a7f5b67d-maas-api-service-ca\") pod \"e123a4ac-0ccf-42c4-82c3-9272a7f5b67d\" (UID: \"e123a4ac-0ccf-42c4-82c3-9272a7f5b67d\") "
Sep 19 12:04:41.400846 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:41.400824    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e123a4ac-0ccf-42c4-82c3-9272a7f5b67d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e123a4ac-0ccf-42c4-82c3-9272a7f5b67d" (UID: "e123a4ac-0ccf-42c4-82c3-9272a7f5b67d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:04:41.502208 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:41.502161    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e123a4ac-0ccf-42c4-82c3-9272a7f5b67d-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:04:42.170919 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:42.170889    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx"
Sep 19 12:04:42.170919 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:42.170908    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx" event={"ID":"e123a4ac-0ccf-42c4-82c3-9272a7f5b67d","Type":"ContainerDied","Data":"39696a860ed2295baeb248985644717eee9586dc714ea78a00d175bb5dc7416f"}
Sep 19 12:04:42.171145 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:42.170948    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39696a860ed2295baeb248985644717eee9586dc714ea78a00d175bb5dc7416f"
Sep 19 12:04:49.401046 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.401011    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"]
Sep 19 12:04:49.401531 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.401265    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k" podUID="ea5fef3d-37b1-423d-bea4-847142a0310c" containerName="payload-processing" containerID="cri-o://1b10cb1c9615f090494769cf8951f855c7c1bbed47180a9e24460d2c80e89d8a" gracePeriod=30
Sep 19 12:04:49.411574 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.411545    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"]
Sep 19 12:04:49.411838 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.411769    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g" podUID="ecfefade-7e7d-466e-bcad-5968111e13d6" containerName="payload-pre-processing" containerID="cri-o://7280a1e38cab6a5e6a53c82049c7c007dcaa2952a953556530f1bb06da8896b7" gracePeriod=30
Sep 19 12:04:49.564259 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.564223    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx"]
Sep 19 12:04:49.566442 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.566416    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-df64a0-m69nx"]
Sep 19 12:04:49.690504 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.690482    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"
Sep 19 12:04:49.693275 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.693260    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"
Sep 19 12:04:49.885574 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.885542    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fm8v\" (UniqueName: \"kubernetes.io/projected/ea5fef3d-37b1-423d-bea4-847142a0310c-kube-api-access-8fm8v\") pod \"ea5fef3d-37b1-423d-bea4-847142a0310c\" (UID: \"ea5fef3d-37b1-423d-bea4-847142a0310c\") "
Sep 19 12:04:49.885751 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.885606    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecfefade-7e7d-466e-bcad-5968111e13d6-plugins-config-volume\") pod \"ecfefade-7e7d-466e-bcad-5968111e13d6\" (UID: \"ecfefade-7e7d-466e-bcad-5968111e13d6\") "
Sep 19 12:04:49.885751 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.885646    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4mjm\" (UniqueName: \"kubernetes.io/projected/ecfefade-7e7d-466e-bcad-5968111e13d6-kube-api-access-p4mjm\") pod \"ecfefade-7e7d-466e-bcad-5968111e13d6\" (UID: \"ecfefade-7e7d-466e-bcad-5968111e13d6\") "
Sep 19 12:04:49.885751 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.885678    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea5fef3d-37b1-423d-bea4-847142a0310c-plugins-config-volume\") pod \"ea5fef3d-37b1-423d-bea4-847142a0310c\" (UID: \"ea5fef3d-37b1-423d-bea4-847142a0310c\") "
Sep 19 12:04:49.886071 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.886044    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea5fef3d-37b1-423d-bea4-847142a0310c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ea5fef3d-37b1-423d-bea4-847142a0310c" (UID: "ea5fef3d-37b1-423d-bea4-847142a0310c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:04:49.886142 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.886044    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecfefade-7e7d-466e-bcad-5968111e13d6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ecfefade-7e7d-466e-bcad-5968111e13d6" (UID: "ecfefade-7e7d-466e-bcad-5968111e13d6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:04:49.887885 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.887858    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea5fef3d-37b1-423d-bea4-847142a0310c-kube-api-access-8fm8v" (OuterVolumeSpecName: "kube-api-access-8fm8v") pod "ea5fef3d-37b1-423d-bea4-847142a0310c" (UID: "ea5fef3d-37b1-423d-bea4-847142a0310c"). InnerVolumeSpecName "kube-api-access-8fm8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:04:49.887885 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.887867    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecfefade-7e7d-466e-bcad-5968111e13d6-kube-api-access-p4mjm" (OuterVolumeSpecName: "kube-api-access-p4mjm") pod "ecfefade-7e7d-466e-bcad-5968111e13d6" (UID: "ecfefade-7e7d-466e-bcad-5968111e13d6"). InnerVolumeSpecName "kube-api-access-p4mjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:04:49.986588 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.986565    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea5fef3d-37b1-423d-bea4-847142a0310c-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:04:49.986588 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.986588    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fm8v\" (UniqueName: \"kubernetes.io/projected/ea5fef3d-37b1-423d-bea4-847142a0310c-kube-api-access-8fm8v\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:04:49.986747 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.986598    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecfefade-7e7d-466e-bcad-5968111e13d6-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:04:49.986747 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:49.986607    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4mjm\" (UniqueName: \"kubernetes.io/projected/ecfefade-7e7d-466e-bcad-5968111e13d6-kube-api-access-p4mjm\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:04:50.216241 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.216211    2559 generic.go:356] "Generic (PLEG): container finished" podID="ea5fef3d-37b1-423d-bea4-847142a0310c" containerID="1b10cb1c9615f090494769cf8951f855c7c1bbed47180a9e24460d2c80e89d8a" exitCode=0
Sep 19 12:04:50.216428 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.216259    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"
Sep 19 12:04:50.216428 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.216303    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k" event={"ID":"ea5fef3d-37b1-423d-bea4-847142a0310c","Type":"ContainerDied","Data":"1b10cb1c9615f090494769cf8951f855c7c1bbed47180a9e24460d2c80e89d8a"}
Sep 19 12:04:50.216428 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.216361    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k" event={"ID":"ea5fef3d-37b1-423d-bea4-847142a0310c","Type":"ContainerDied","Data":"7cbfa55ea5b487321b9d9a42aa2c071b5dcc6c7e798a4e3ffb151092926699a9"}
Sep 19 12:04:50.216428 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.216394    2559 scope.go:117] "RemoveContainer" containerID="1b10cb1c9615f090494769cf8951f855c7c1bbed47180a9e24460d2c80e89d8a"
Sep 19 12:04:50.217710 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.217686    2559 generic.go:356] "Generic (PLEG): container finished" podID="ecfefade-7e7d-466e-bcad-5968111e13d6" containerID="7280a1e38cab6a5e6a53c82049c7c007dcaa2952a953556530f1bb06da8896b7" exitCode=0
Sep 19 12:04:50.217799 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.217745    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"
Sep 19 12:04:50.217799 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.217759    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g" event={"ID":"ecfefade-7e7d-466e-bcad-5968111e13d6","Type":"ContainerDied","Data":"7280a1e38cab6a5e6a53c82049c7c007dcaa2952a953556530f1bb06da8896b7"}
Sep 19 12:04:50.217903 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.217802    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g" event={"ID":"ecfefade-7e7d-466e-bcad-5968111e13d6","Type":"ContainerDied","Data":"5d4c26bbfd7860ff728a94a8161de5c4a6bfc448cf9dd47355628c71439fae93"}
Sep 19 12:04:50.226696 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.226680    2559 scope.go:117] "RemoveContainer" containerID="1b10cb1c9615f090494769cf8951f855c7c1bbed47180a9e24460d2c80e89d8a"
Sep 19 12:04:50.226963 ip-10-0-140-136 kubenswrapper[2559]: E0919 12:04:50.226943    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b10cb1c9615f090494769cf8951f855c7c1bbed47180a9e24460d2c80e89d8a\": container with ID starting with 1b10cb1c9615f090494769cf8951f855c7c1bbed47180a9e24460d2c80e89d8a not found: ID does not exist" containerID="1b10cb1c9615f090494769cf8951f855c7c1bbed47180a9e24460d2c80e89d8a"
Sep 19 12:04:50.227055 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.226972    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b10cb1c9615f090494769cf8951f855c7c1bbed47180a9e24460d2c80e89d8a"} err="failed to get container status \"1b10cb1c9615f090494769cf8951f855c7c1bbed47180a9e24460d2c80e89d8a\": rpc error: code = NotFound desc = could not find container \"1b10cb1c9615f090494769cf8951f855c7c1bbed47180a9e24460d2c80e89d8a\": container with ID starting with 1b10cb1c9615f090494769cf8951f855c7c1bbed47180a9e24460d2c80e89d8a not found: ID does not exist"
Sep 19 12:04:50.227055 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.226992    2559 scope.go:117] "RemoveContainer" containerID="7280a1e38cab6a5e6a53c82049c7c007dcaa2952a953556530f1bb06da8896b7"
Sep 19 12:04:50.235538 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.235521    2559 scope.go:117] "RemoveContainer" containerID="7280a1e38cab6a5e6a53c82049c7c007dcaa2952a953556530f1bb06da8896b7"
Sep 19 12:04:50.235792 ip-10-0-140-136 kubenswrapper[2559]: E0919 12:04:50.235774    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7280a1e38cab6a5e6a53c82049c7c007dcaa2952a953556530f1bb06da8896b7\": container with ID starting with 7280a1e38cab6a5e6a53c82049c7c007dcaa2952a953556530f1bb06da8896b7 not found: ID does not exist" containerID="7280a1e38cab6a5e6a53c82049c7c007dcaa2952a953556530f1bb06da8896b7"
Sep 19 12:04:50.235852 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.235803    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7280a1e38cab6a5e6a53c82049c7c007dcaa2952a953556530f1bb06da8896b7"} err="failed to get container status \"7280a1e38cab6a5e6a53c82049c7c007dcaa2952a953556530f1bb06da8896b7\": rpc error: code = NotFound desc = could not find container \"7280a1e38cab6a5e6a53c82049c7c007dcaa2952a953556530f1bb06da8896b7\": container with ID starting with 7280a1e38cab6a5e6a53c82049c7c007dcaa2952a953556530f1bb06da8896b7 not found: ID does not exist"
Sep 19 12:04:50.241022 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.240967    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"]
Sep 19 12:04:50.246717 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.246697    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-df64a0-7856c477cd-2tl6g"]
Sep 19 12:04:50.256819 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.256799    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"]
Sep 19 12:04:50.258845 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.258828    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-df64a0-648c464bdf-kdg4k"]
Sep 19 12:04:50.595406 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.595327    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e123a4ac-0ccf-42c4-82c3-9272a7f5b67d" path="/var/lib/kubelet/pods/e123a4ac-0ccf-42c4-82c3-9272a7f5b67d/volumes"
Sep 19 12:04:50.595732 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.595677    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea5fef3d-37b1-423d-bea4-847142a0310c" path="/var/lib/kubelet/pods/ea5fef3d-37b1-423d-bea4-847142a0310c/volumes"
Sep 19 12:04:50.595983 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:50.595970    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecfefade-7e7d-466e-bcad-5968111e13d6" path="/var/lib/kubelet/pods/ecfefade-7e7d-466e-bcad-5968111e13d6/volumes"
Sep 19 12:04:56.481172 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.481133    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j"]
Sep 19 12:04:56.481678 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.481660    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecfefade-7e7d-466e-bcad-5968111e13d6" containerName="payload-pre-processing"
Sep 19 12:04:56.481731 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.481680    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecfefade-7e7d-466e-bcad-5968111e13d6" containerName="payload-pre-processing"
Sep 19 12:04:56.481731 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.481696    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e123a4ac-0ccf-42c4-82c3-9272a7f5b67d" containerName="revoke-keys"
Sep 19 12:04:56.481731 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.481704    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e123a4ac-0ccf-42c4-82c3-9272a7f5b67d" containerName="revoke-keys"
Sep 19 12:04:56.481731 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.481713    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea5fef3d-37b1-423d-bea4-847142a0310c" containerName="payload-processing"
Sep 19 12:04:56.481731 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.481719    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea5fef3d-37b1-423d-bea4-847142a0310c" containerName="payload-processing"
Sep 19 12:04:56.481912 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.481786    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecfefade-7e7d-466e-bcad-5968111e13d6" containerName="payload-pre-processing"
Sep 19 12:04:56.481912 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.481795    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e123a4ac-0ccf-42c4-82c3-9272a7f5b67d" containerName="revoke-keys"
Sep 19 12:04:56.481912 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.481806    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea5fef3d-37b1-423d-bea4-847142a0310c" containerName="payload-processing"
Sep 19 12:04:56.484894 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.484878    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j"
Sep 19 12:04:56.487438 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.487416    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 12:04:56.491619 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.491593    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j"]
Sep 19 12:04:56.548432 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.548406    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91b8e8a8-762a-4ad5-85c4-3c01e2308351-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j\" (UID: \"91b8e8a8-762a-4ad5-85c4-3c01e2308351\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j"
Sep 19 12:04:56.649775 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.649737    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91b8e8a8-762a-4ad5-85c4-3c01e2308351-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j\" (UID: \"91b8e8a8-762a-4ad5-85c4-3c01e2308351\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j"
Sep 19 12:04:56.650344 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.650326    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91b8e8a8-762a-4ad5-85c4-3c01e2308351-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j\" (UID: \"91b8e8a8-762a-4ad5-85c4-3c01e2308351\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j"
Sep 19 12:04:56.797955 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:56.797860    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j"
Sep 19 12:04:57.068021 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:57.067990    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j"]
Sep 19 12:04:57.069748 ip-10-0-140-136 kubenswrapper[2559]: W0919 12:04:57.069724    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b8e8a8_762a_4ad5_85c4_3c01e2308351.slice/crio-0f11e94035e1c2fda4820ffc4386ef739fac9dbfa39f1f3e94621fc90c75d9b5 WatchSource:0}: Error finding container 0f11e94035e1c2fda4820ffc4386ef739fac9dbfa39f1f3e94621fc90c75d9b5: Status 404 returned error can't find the container with id 0f11e94035e1c2fda4820ffc4386ef739fac9dbfa39f1f3e94621fc90c75d9b5
Sep 19 12:04:57.257787 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:57.257753    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j" event={"ID":"91b8e8a8-762a-4ad5-85c4-3c01e2308351","Type":"ContainerStarted","Data":"0f11e94035e1c2fda4820ffc4386ef739fac9dbfa39f1f3e94621fc90c75d9b5"}
Sep 19 12:04:58.265148 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:58.265113    2559 generic.go:356] "Generic (PLEG): container finished" podID="91b8e8a8-762a-4ad5-85c4-3c01e2308351" containerID="3d2ed75d0b20390c89a0aaf1aa123e868601bcdf2d9f808061ee2053dac2598c" exitCode=0
Sep 19 12:04:58.265655 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:58.265213    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j" event={"ID":"91b8e8a8-762a-4ad5-85c4-3c01e2308351","Type":"ContainerDied","Data":"3d2ed75d0b20390c89a0aaf1aa123e868601bcdf2d9f808061ee2053dac2598c"}
Sep 19 12:04:59.404809 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:59.404779    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j"
Sep 19 12:04:59.478648 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:59.478621    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91b8e8a8-762a-4ad5-85c4-3c01e2308351-maas-api-service-ca\") pod \"91b8e8a8-762a-4ad5-85c4-3c01e2308351\" (UID: \"91b8e8a8-762a-4ad5-85c4-3c01e2308351\") "
Sep 19 12:04:59.478933 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:59.478912    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91b8e8a8-762a-4ad5-85c4-3c01e2308351-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "91b8e8a8-762a-4ad5-85c4-3c01e2308351" (UID: "91b8e8a8-762a-4ad5-85c4-3c01e2308351"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:04:59.579618 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:04:59.579548    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91b8e8a8-762a-4ad5-85c4-3c01e2308351-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:00.277682 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:00.277652    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j"
Sep 19 12:05:00.277836 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:00.277653    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j" event={"ID":"91b8e8a8-762a-4ad5-85c4-3c01e2308351","Type":"ContainerDied","Data":"0f11e94035e1c2fda4820ffc4386ef739fac9dbfa39f1f3e94621fc90c75d9b5"}
Sep 19 12:05:00.277836 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:00.277792    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f11e94035e1c2fda4820ffc4386ef739fac9dbfa39f1f3e94621fc90c75d9b5"
Sep 19 12:05:07.101353 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:07.101312    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"]
Sep 19 12:05:07.101921 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:07.101564    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p" podUID="3c12cd77-c733-48ed-9a3f-71db94cc08fc" containerName="payload-pre-processing" containerID="cri-o://dee3390ddf741b8d0ad7d9c7818f13688af745c822cd486736d3de1a97b2adf8" gracePeriod=30
Sep 19 12:05:07.247498 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:07.247455    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j"]
Sep 19 12:05:07.250352 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:07.250310    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-4937ff-ct77j"]
Sep 19 12:05:07.330060 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:07.330027    2559 generic.go:356] "Generic (PLEG): container finished" podID="3c12cd77-c733-48ed-9a3f-71db94cc08fc" containerID="dee3390ddf741b8d0ad7d9c7818f13688af745c822cd486736d3de1a97b2adf8" exitCode=0
Sep 19 12:05:07.330244 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:07.330110    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p" event={"ID":"3c12cd77-c733-48ed-9a3f-71db94cc08fc","Type":"ContainerDied","Data":"dee3390ddf741b8d0ad7d9c7818f13688af745c822cd486736d3de1a97b2adf8"}
Sep 19 12:05:07.373097 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:07.373025    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"
Sep 19 12:05:07.450486 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:07.450456    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c12cd77-c733-48ed-9a3f-71db94cc08fc-plugins-config-volume\") pod \"3c12cd77-c733-48ed-9a3f-71db94cc08fc\" (UID: \"3c12cd77-c733-48ed-9a3f-71db94cc08fc\") "
Sep 19 12:05:07.450641 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:07.450526    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlcrj\" (UniqueName: \"kubernetes.io/projected/3c12cd77-c733-48ed-9a3f-71db94cc08fc-kube-api-access-dlcrj\") pod \"3c12cd77-c733-48ed-9a3f-71db94cc08fc\" (UID: \"3c12cd77-c733-48ed-9a3f-71db94cc08fc\") "
Sep 19 12:05:07.450775 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:07.450752    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c12cd77-c733-48ed-9a3f-71db94cc08fc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3c12cd77-c733-48ed-9a3f-71db94cc08fc" (UID: "3c12cd77-c733-48ed-9a3f-71db94cc08fc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:05:07.452670 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:07.452644    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c12cd77-c733-48ed-9a3f-71db94cc08fc-kube-api-access-dlcrj" (OuterVolumeSpecName: "kube-api-access-dlcrj") pod "3c12cd77-c733-48ed-9a3f-71db94cc08fc" (UID: "3c12cd77-c733-48ed-9a3f-71db94cc08fc"). InnerVolumeSpecName "kube-api-access-dlcrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:05:07.551809 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:07.551780    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c12cd77-c733-48ed-9a3f-71db94cc08fc-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:07.551809 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:07.551805    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlcrj\" (UniqueName: \"kubernetes.io/projected/3c12cd77-c733-48ed-9a3f-71db94cc08fc-kube-api-access-dlcrj\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:08.336824 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:08.336792    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p" event={"ID":"3c12cd77-c733-48ed-9a3f-71db94cc08fc","Type":"ContainerDied","Data":"e6d0e081251c37d7679c72c70d60d17a56299035ba51517ebaa2d8cd918097a6"}
Sep 19 12:05:08.337262 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:08.336842    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"
Sep 19 12:05:08.337262 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:08.336853    2559 scope.go:117] "RemoveContainer" containerID="dee3390ddf741b8d0ad7d9c7818f13688af745c822cd486736d3de1a97b2adf8"
Sep 19 12:05:08.359427 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:08.359402    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"]
Sep 19 12:05:08.363204 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:08.363173    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-4937ff-77b8f95b54-bjw9p"]
Sep 19 12:05:08.601809 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:08.601731    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c12cd77-c733-48ed-9a3f-71db94cc08fc" path="/var/lib/kubelet/pods/3c12cd77-c733-48ed-9a3f-71db94cc08fc/volumes"
Sep 19 12:05:08.602048 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:08.602035    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91b8e8a8-762a-4ad5-85c4-3c01e2308351" path="/var/lib/kubelet/pods/91b8e8a8-762a-4ad5-85c4-3c01e2308351/volumes"
Sep 19 12:05:13.701776 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:13.701741    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"]
Sep 19 12:05:13.702334 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:13.701962    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x" podUID="4afed85b-980b-4c82-b7ea-894e3aee992a" containerName="istio-proxy" containerID="cri-o://85969c55fca49fd2cac55711384fe8973013012eb5a219b73137cfa4a43a6fb1" gracePeriod=30
Sep 19 12:05:14.074483 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:14.074406    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw"]
Sep 19 12:05:14.074835 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:14.074822    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91b8e8a8-762a-4ad5-85c4-3c01e2308351" containerName="revoke-keys"
Sep 19 12:05:14.074887 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:14.074839    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="91b8e8a8-762a-4ad5-85c4-3c01e2308351" containerName="revoke-keys"
Sep 19 12:05:14.074887 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:14.074861    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c12cd77-c733-48ed-9a3f-71db94cc08fc" containerName="payload-pre-processing"
Sep 19 12:05:14.074887 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:14.074867    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c12cd77-c733-48ed-9a3f-71db94cc08fc" containerName="payload-pre-processing"
Sep 19 12:05:14.074985 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:14.074936    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="91b8e8a8-762a-4ad5-85c4-3c01e2308351" containerName="revoke-keys"
Sep 19 12:05:14.074985 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:14.074947    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c12cd77-c733-48ed-9a3f-71db94cc08fc" containerName="payload-pre-processing"
Sep 19 12:05:14.078093 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:14.078076    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw"
Sep 19 12:05:14.080746 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:14.080722    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bnlmc\""
Sep 19 12:05:14.097395 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:14.097350    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw"]
Sep 19 12:05:14.215160 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:14.215109    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/010e5d08-cad9-43a9-9c7d-1041b58f24e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw\" (UID: \"010e5d08-cad9-43a9-9c7d-1041b58f24e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw"
Sep 19 12:05:14.316607 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:14.316568    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/010e5d08-cad9-43a9-9c7d-1041b58f24e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw\" (UID: \"010e5d08-cad9-43a9-9c7d-1041b58f24e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw"
Sep 19 12:05:14.317257 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:14.317221    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/010e5d08-cad9-43a9-9c7d-1041b58f24e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw\" (UID: \"010e5d08-cad9-43a9-9c7d-1041b58f24e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw"
Sep 19 12:05:14.398791 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:14.398705    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw"
Sep 19 12:05:14.673092 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:14.673064    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw"]
Sep 19 12:05:14.675564 ip-10-0-140-136 kubenswrapper[2559]: W0919 12:05:14.675536    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod010e5d08_cad9_43a9_9c7d_1041b58f24e7.slice/crio-a0bacad6af10056d9be99707f1a47e1fe1f31f484d9141d6a47adba9e5b33955 WatchSource:0}: Error finding container a0bacad6af10056d9be99707f1a47e1fe1f31f484d9141d6a47adba9e5b33955: Status 404 returned error can't find the container with id a0bacad6af10056d9be99707f1a47e1fe1f31f484d9141d6a47adba9e5b33955
Sep 19 12:05:15.375855 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:15.375818    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw" event={"ID":"010e5d08-cad9-43a9-9c7d-1041b58f24e7","Type":"ContainerStarted","Data":"d0a43798d7661d7c8c9089ef00f549ed4cca83a99c86cbb3b9435d6ef72eec57"}
Sep 19 12:05:15.376274 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:15.375881    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw" event={"ID":"010e5d08-cad9-43a9-9c7d-1041b58f24e7","Type":"ContainerStarted","Data":"a0bacad6af10056d9be99707f1a47e1fe1f31f484d9141d6a47adba9e5b33955"}
Sep 19 12:05:15.391056 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:15.391013    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw" podStartSLOduration=1.390996453 podStartE2EDuration="1.390996453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:05:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:05:15.389242922 +0000 UTC m=+1867.311555804" watchObservedRunningTime="2026-09-19 12:05:15.390996453 +0000 UTC m=+1867.313309333"
Sep 19 12:05:16.382659 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:16.382629    2559 generic.go:356] "Generic (PLEG): container finished" podID="010e5d08-cad9-43a9-9c7d-1041b58f24e7" containerID="d0a43798d7661d7c8c9089ef00f549ed4cca83a99c86cbb3b9435d6ef72eec57" exitCode=0
Sep 19 12:05:16.383051 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:16.382726    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw" event={"ID":"010e5d08-cad9-43a9-9c7d-1041b58f24e7","Type":"ContainerDied","Data":"d0a43798d7661d7c8c9089ef00f549ed4cca83a99c86cbb3b9435d6ef72eec57"}
Sep 19 12:05:17.519236 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:17.519212    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw"
Sep 19 12:05:17.647082 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:17.647009    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/010e5d08-cad9-43a9-9c7d-1041b58f24e7-maas-api-service-ca\") pod \"010e5d08-cad9-43a9-9c7d-1041b58f24e7\" (UID: \"010e5d08-cad9-43a9-9c7d-1041b58f24e7\") "
Sep 19 12:05:17.647352 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:17.647333    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/010e5d08-cad9-43a9-9c7d-1041b58f24e7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "010e5d08-cad9-43a9-9c7d-1041b58f24e7" (UID: "010e5d08-cad9-43a9-9c7d-1041b58f24e7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:05:17.748369 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:17.748340    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/010e5d08-cad9-43a9-9c7d-1041b58f24e7-maas-api-service-ca\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:18.394489 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:18.394462    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw"
Sep 19 12:05:18.394654 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:18.394502    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw" event={"ID":"010e5d08-cad9-43a9-9c7d-1041b58f24e7","Type":"ContainerDied","Data":"a0bacad6af10056d9be99707f1a47e1fe1f31f484d9141d6a47adba9e5b33955"}
Sep 19 12:05:18.394654 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:18.394548    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0bacad6af10056d9be99707f1a47e1fe1f31f484d9141d6a47adba9e5b33955"
Sep 19 12:05:18.949690 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:18.949665    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 12:05:19.060683 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.060607    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-token\") pod \"4afed85b-980b-4c82-b7ea-894e3aee992a\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") "
Sep 19 12:05:19.060683 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.060648    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7v6cb\" (UniqueName: \"kubernetes.io/projected/4afed85b-980b-4c82-b7ea-894e3aee992a-kube-api-access-7v6cb\") pod \"4afed85b-980b-4c82-b7ea-894e3aee992a\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") "
Sep 19 12:05:19.060683 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.060681    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-credential-socket\") pod \"4afed85b-980b-4c82-b7ea-894e3aee992a\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") "
Sep 19 12:05:19.060959 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.060732    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-data\") pod \"4afed85b-980b-4c82-b7ea-894e3aee992a\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") "
Sep 19 12:05:19.060959 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.060767    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-workload-socket\") pod \"4afed85b-980b-4c82-b7ea-894e3aee992a\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") "
Sep 19 12:05:19.060959 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.060821    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-podinfo\") pod \"4afed85b-980b-4c82-b7ea-894e3aee992a\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") "
Sep 19 12:05:19.060959 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.060858    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4afed85b-980b-4c82-b7ea-894e3aee992a-istiod-ca-cert\") pod \"4afed85b-980b-4c82-b7ea-894e3aee992a\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") "
Sep 19 12:05:19.060959 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.060952    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-envoy\") pod \"4afed85b-980b-4c82-b7ea-894e3aee992a\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") "
Sep 19 12:05:19.061255 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.060981    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-workload-certs\") pod \"4afed85b-980b-4c82-b7ea-894e3aee992a\" (UID: \"4afed85b-980b-4c82-b7ea-894e3aee992a\") "
Sep 19 12:05:19.061255 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.061049    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4afed85b-980b-4c82-b7ea-894e3aee992a" (UID: "4afed85b-980b-4c82-b7ea-894e3aee992a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:05:19.061255 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.061065    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-data" (OuterVolumeSpecName: "istio-data") pod "4afed85b-980b-4c82-b7ea-894e3aee992a" (UID: "4afed85b-980b-4c82-b7ea-894e3aee992a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:05:19.061255 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.061099    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4afed85b-980b-4c82-b7ea-894e3aee992a" (UID: "4afed85b-980b-4c82-b7ea-894e3aee992a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:05:19.061477 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.061328    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4afed85b-980b-4c82-b7ea-894e3aee992a" (UID: "4afed85b-980b-4c82-b7ea-894e3aee992a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:05:19.061477 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.061425    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4afed85b-980b-4c82-b7ea-894e3aee992a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4afed85b-980b-4c82-b7ea-894e3aee992a" (UID: "4afed85b-980b-4c82-b7ea-894e3aee992a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:05:19.061597 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.061529    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-credential-socket\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:19.061597 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.061548    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-data\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:19.061597 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.061562    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-workload-socket\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:19.061597 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.061577    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4afed85b-980b-4c82-b7ea-894e3aee992a-istiod-ca-cert\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:19.061597 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.061590    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-workload-certs\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:19.063047 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.063023    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4afed85b-980b-4c82-b7ea-894e3aee992a" (UID: "4afed85b-980b-4c82-b7ea-894e3aee992a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:05:19.063136 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.063099    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-token" (OuterVolumeSpecName: "istio-token") pod "4afed85b-980b-4c82-b7ea-894e3aee992a" (UID: "4afed85b-980b-4c82-b7ea-894e3aee992a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:05:19.063527 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.063502    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4afed85b-980b-4c82-b7ea-894e3aee992a-kube-api-access-7v6cb" (OuterVolumeSpecName: "kube-api-access-7v6cb") pod "4afed85b-980b-4c82-b7ea-894e3aee992a" (UID: "4afed85b-980b-4c82-b7ea-894e3aee992a"). InnerVolumeSpecName "kube-api-access-7v6cb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:05:19.063527 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.063503    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4afed85b-980b-4c82-b7ea-894e3aee992a" (UID: "4afed85b-980b-4c82-b7ea-894e3aee992a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:05:19.162021 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.161995    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-podinfo\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:19.162021 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.162017    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-envoy\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:19.162021 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.162027    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4afed85b-980b-4c82-b7ea-894e3aee992a-istio-token\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:19.162234 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.162035    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7v6cb\" (UniqueName: \"kubernetes.io/projected/4afed85b-980b-4c82-b7ea-894e3aee992a-kube-api-access-7v6cb\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:19.401809 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.401711    2559 generic.go:356] "Generic (PLEG): container finished" podID="4afed85b-980b-4c82-b7ea-894e3aee992a" containerID="85969c55fca49fd2cac55711384fe8973013012eb5a219b73137cfa4a43a6fb1" exitCode=0
Sep 19 12:05:19.401809 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.401762    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"
Sep 19 12:05:19.402040 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.401759    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x" event={"ID":"4afed85b-980b-4c82-b7ea-894e3aee992a","Type":"ContainerDied","Data":"85969c55fca49fd2cac55711384fe8973013012eb5a219b73137cfa4a43a6fb1"}
Sep 19 12:05:19.402040 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.401885    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x" event={"ID":"4afed85b-980b-4c82-b7ea-894e3aee992a","Type":"ContainerDied","Data":"e64377258aeddce3a60ebbff93fed0fe743ddc159213c413ec923aeefb7c1227"}
Sep 19 12:05:19.402040 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.401909    2559 scope.go:117] "RemoveContainer" containerID="85969c55fca49fd2cac55711384fe8973013012eb5a219b73137cfa4a43a6fb1"
Sep 19 12:05:19.411720 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.411705    2559 scope.go:117] "RemoveContainer" containerID="e8f9a8ab29aa9b37013cac4cb35c41ddc7b546bbf70504634fcc0a5d0d7bb00a"
Sep 19 12:05:19.419667 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.419644    2559 scope.go:117] "RemoveContainer" containerID="85969c55fca49fd2cac55711384fe8973013012eb5a219b73137cfa4a43a6fb1"
Sep 19 12:05:19.419949 ip-10-0-140-136 kubenswrapper[2559]: E0919 12:05:19.419931    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85969c55fca49fd2cac55711384fe8973013012eb5a219b73137cfa4a43a6fb1\": container with ID starting with 85969c55fca49fd2cac55711384fe8973013012eb5a219b73137cfa4a43a6fb1 not found: ID does not exist" containerID="85969c55fca49fd2cac55711384fe8973013012eb5a219b73137cfa4a43a6fb1"
Sep 19 12:05:19.420015 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.419956    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85969c55fca49fd2cac55711384fe8973013012eb5a219b73137cfa4a43a6fb1"} err="failed to get container status \"85969c55fca49fd2cac55711384fe8973013012eb5a219b73137cfa4a43a6fb1\": rpc error: code = NotFound desc = could not find container \"85969c55fca49fd2cac55711384fe8973013012eb5a219b73137cfa4a43a6fb1\": container with ID starting with 85969c55fca49fd2cac55711384fe8973013012eb5a219b73137cfa4a43a6fb1 not found: ID does not exist"
Sep 19 12:05:19.420015 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.419972    2559 scope.go:117] "RemoveContainer" containerID="e8f9a8ab29aa9b37013cac4cb35c41ddc7b546bbf70504634fcc0a5d0d7bb00a"
Sep 19 12:05:19.420350 ip-10-0-140-136 kubenswrapper[2559]: E0919 12:05:19.420283    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8f9a8ab29aa9b37013cac4cb35c41ddc7b546bbf70504634fcc0a5d0d7bb00a\": container with ID starting with e8f9a8ab29aa9b37013cac4cb35c41ddc7b546bbf70504634fcc0a5d0d7bb00a not found: ID does not exist" containerID="e8f9a8ab29aa9b37013cac4cb35c41ddc7b546bbf70504634fcc0a5d0d7bb00a"
Sep 19 12:05:19.420350 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.420326    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8f9a8ab29aa9b37013cac4cb35c41ddc7b546bbf70504634fcc0a5d0d7bb00a"} err="failed to get container status \"e8f9a8ab29aa9b37013cac4cb35c41ddc7b546bbf70504634fcc0a5d0d7bb00a\": rpc error: code = NotFound desc = could not find container \"e8f9a8ab29aa9b37013cac4cb35c41ddc7b546bbf70504634fcc0a5d0d7bb00a\": container with ID starting with e8f9a8ab29aa9b37013cac4cb35c41ddc7b546bbf70504634fcc0a5d0d7bb00a not found: ID does not exist"
Sep 19 12:05:19.423707 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.423686    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"]
Sep 19 12:05:19.426934 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:19.426914    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-4937ff-openshift-default-68d5674cfd-lm86x"]
Sep 19 12:05:20.595177 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:20.595141    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4afed85b-980b-4c82-b7ea-894e3aee992a" path="/var/lib/kubelet/pods/4afed85b-980b-4c82-b7ea-894e3aee992a/volumes"
Sep 19 12:05:24.541265 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:24.541229    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-4a8a59-29830320-6zwps"]
Sep 19 12:05:24.547415 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:24.547383    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-4a8a59-29830320-6zwps"]
Sep 19 12:05:24.575254 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:24.575223    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"]
Sep 19 12:05:24.575453 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:24.575427    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft" podUID="76519908-fac0-4508-b417-78423417cf34" containerName="payload-pre-processing" containerID="cri-o://c710b5837e9ff144100671de7fac21f44ea9eb55e9cbc2389adce7b8565e1785" gracePeriod=30
Sep 19 12:05:24.600019 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:24.599986    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca3c2846-9a6f-4284-aadc-cdede6f1e706" path="/var/lib/kubelet/pods/ca3c2846-9a6f-4284-aadc-cdede6f1e706/volumes"
Sep 19 12:05:24.720299 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:24.720268    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw"]
Sep 19 12:05:24.723547 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:24.723522    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4a8a59-5psgw"]
Sep 19 12:05:24.837469 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:24.837443    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"
Sep 19 12:05:25.019932 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:25.019897    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8br4p\" (UniqueName: \"kubernetes.io/projected/76519908-fac0-4508-b417-78423417cf34-kube-api-access-8br4p\") pod \"76519908-fac0-4508-b417-78423417cf34\" (UID: \"76519908-fac0-4508-b417-78423417cf34\") "
Sep 19 12:05:25.020101 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:25.019955    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76519908-fac0-4508-b417-78423417cf34-plugins-config-volume\") pod \"76519908-fac0-4508-b417-78423417cf34\" (UID: \"76519908-fac0-4508-b417-78423417cf34\") "
Sep 19 12:05:25.020309 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:25.020286    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76519908-fac0-4508-b417-78423417cf34-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "76519908-fac0-4508-b417-78423417cf34" (UID: "76519908-fac0-4508-b417-78423417cf34"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:05:25.022251 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:25.022230    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76519908-fac0-4508-b417-78423417cf34-kube-api-access-8br4p" (OuterVolumeSpecName: "kube-api-access-8br4p") pod "76519908-fac0-4508-b417-78423417cf34" (UID: "76519908-fac0-4508-b417-78423417cf34"). InnerVolumeSpecName "kube-api-access-8br4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:05:25.120901 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:25.120817    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8br4p\" (UniqueName: \"kubernetes.io/projected/76519908-fac0-4508-b417-78423417cf34-kube-api-access-8br4p\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:25.120901 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:25.120852    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76519908-fac0-4508-b417-78423417cf34-plugins-config-volume\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:25.435796 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:25.435763    2559 generic.go:356] "Generic (PLEG): container finished" podID="76519908-fac0-4508-b417-78423417cf34" containerID="c710b5837e9ff144100671de7fac21f44ea9eb55e9cbc2389adce7b8565e1785" exitCode=0
Sep 19 12:05:25.435967 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:25.435804    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"
Sep 19 12:05:25.435967 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:25.435812    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft" event={"ID":"76519908-fac0-4508-b417-78423417cf34","Type":"ContainerDied","Data":"c710b5837e9ff144100671de7fac21f44ea9eb55e9cbc2389adce7b8565e1785"}
Sep 19 12:05:25.435967 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:25.435883    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft" event={"ID":"76519908-fac0-4508-b417-78423417cf34","Type":"ContainerDied","Data":"dbd32cbeffbbfd51ba6ba6713db877deb9df9282ceec10e9862d3f478f8c8125"}
Sep 19 12:05:25.435967 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:25.435914    2559 scope.go:117] "RemoveContainer" containerID="c710b5837e9ff144100671de7fac21f44ea9eb55e9cbc2389adce7b8565e1785"
Sep 19 12:05:25.445581 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:25.445560    2559 scope.go:117] "RemoveContainer" containerID="c710b5837e9ff144100671de7fac21f44ea9eb55e9cbc2389adce7b8565e1785"
Sep 19 12:05:25.445849 ip-10-0-140-136 kubenswrapper[2559]: E0919 12:05:25.445828    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c710b5837e9ff144100671de7fac21f44ea9eb55e9cbc2389adce7b8565e1785\": container with ID starting with c710b5837e9ff144100671de7fac21f44ea9eb55e9cbc2389adce7b8565e1785 not found: ID does not exist" containerID="c710b5837e9ff144100671de7fac21f44ea9eb55e9cbc2389adce7b8565e1785"
Sep 19 12:05:25.445918 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:25.445856    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c710b5837e9ff144100671de7fac21f44ea9eb55e9cbc2389adce7b8565e1785"} err="failed to get container status \"c710b5837e9ff144100671de7fac21f44ea9eb55e9cbc2389adce7b8565e1785\": rpc error: code = NotFound desc = could not find container \"c710b5837e9ff144100671de7fac21f44ea9eb55e9cbc2389adce7b8565e1785\": container with ID starting with c710b5837e9ff144100671de7fac21f44ea9eb55e9cbc2389adce7b8565e1785 not found: ID does not exist"
Sep 19 12:05:25.456949 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:25.456926    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"]
Sep 19 12:05:25.460482 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:25.460460    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4a8a59-848f7b547d-dd5ft"]
Sep 19 12:05:26.595740 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:26.595706    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="010e5d08-cad9-43a9-9c7d-1041b58f24e7" path="/var/lib/kubelet/pods/010e5d08-cad9-43a9-9c7d-1041b58f24e7/volumes"
Sep 19 12:05:26.596116 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:26.596060    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76519908-fac0-4508-b417-78423417cf34" path="/var/lib/kubelet/pods/76519908-fac0-4508-b417-78423417cf34/volumes"
Sep 19 12:05:31.121458 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:31.121366    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"]
Sep 19 12:05:31.121886 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:31.121661    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl" podUID="6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" containerName="istio-proxy" containerID="cri-o://9290649dd999dbcc3a55413835a677c7b4b0885da4d1227c19c2086069e9b283" gracePeriod=30
Sep 19 12:05:36.373076 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.373048    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 12:05:36.423472 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.423405    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-workload-socket\") pod \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") "
Sep 19 12:05:36.423472 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.423442    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-data\") pod \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") "
Sep 19 12:05:36.423472 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.423461    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-workload-certs\") pod \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") "
Sep 19 12:05:36.423737 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.423478    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-credential-socket\") pod \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") "
Sep 19 12:05:36.423737 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.423514    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-podinfo\") pod \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") "
Sep 19 12:05:36.423737 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.423541    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-envoy\") pod \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") "
Sep 19 12:05:36.423737 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.423561    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istiod-ca-cert\") pod \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") "
Sep 19 12:05:36.423737 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.423583    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7672\" (UniqueName: \"kubernetes.io/projected/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-kube-api-access-k7672\") pod \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") "
Sep 19 12:05:36.423737 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.423602    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-token\") pod \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\" (UID: \"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85\") "
Sep 19 12:05:36.423737 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.423691    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-data" (OuterVolumeSpecName: "istio-data") pod "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" (UID: "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:05:36.424101 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.423791    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" (UID: "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:05:36.424101 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.423882    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" (UID: "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:05:36.424101 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.423950    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-data\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:36.424101 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.423969    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-workload-certs\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:36.424101 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.423989    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" (UID: "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:05:36.424101 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.424071    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" (UID: "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:05:36.425946 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.425919    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" (UID: "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:05:36.426031 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.425942    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-kube-api-access-k7672" (OuterVolumeSpecName: "kube-api-access-k7672") pod "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" (UID: "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85"). InnerVolumeSpecName "kube-api-access-k7672". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:05:36.426243 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.426224    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" (UID: "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:05:36.426295 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.426251    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-token" (OuterVolumeSpecName: "istio-token") pod "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" (UID: "6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:05:36.495541 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.495513    2559 generic.go:356] "Generic (PLEG): container finished" podID="6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" containerID="9290649dd999dbcc3a55413835a677c7b4b0885da4d1227c19c2086069e9b283" exitCode=0
Sep 19 12:05:36.495647 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.495561    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"
Sep 19 12:05:36.495647 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.495565    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl" event={"ID":"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85","Type":"ContainerDied","Data":"9290649dd999dbcc3a55413835a677c7b4b0885da4d1227c19c2086069e9b283"}
Sep 19 12:05:36.495647 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.495607    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl" event={"ID":"6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85","Type":"ContainerDied","Data":"2c38d6d57c4a857d1ae4895eea630a7cbca00b4b1ca68ac000945b78ca986da1"}
Sep 19 12:05:36.495647 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.495628    2559 scope.go:117] "RemoveContainer" containerID="9290649dd999dbcc3a55413835a677c7b4b0885da4d1227c19c2086069e9b283"
Sep 19 12:05:36.504533 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.504516    2559 scope.go:117] "RemoveContainer" containerID="ad763b6097f66b6838d3af3255275e859f8c885eee504ecb13d71bb4732c29bc"
Sep 19 12:05:36.512697 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.512678    2559 scope.go:117] "RemoveContainer" containerID="9290649dd999dbcc3a55413835a677c7b4b0885da4d1227c19c2086069e9b283"
Sep 19 12:05:36.512955 ip-10-0-140-136 kubenswrapper[2559]: E0919 12:05:36.512936    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9290649dd999dbcc3a55413835a677c7b4b0885da4d1227c19c2086069e9b283\": container with ID starting with 9290649dd999dbcc3a55413835a677c7b4b0885da4d1227c19c2086069e9b283 not found: ID does not exist" containerID="9290649dd999dbcc3a55413835a677c7b4b0885da4d1227c19c2086069e9b283"
Sep 19 12:05:36.513015 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.512961    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9290649dd999dbcc3a55413835a677c7b4b0885da4d1227c19c2086069e9b283"} err="failed to get container status \"9290649dd999dbcc3a55413835a677c7b4b0885da4d1227c19c2086069e9b283\": rpc error: code = NotFound desc = could not find container \"9290649dd999dbcc3a55413835a677c7b4b0885da4d1227c19c2086069e9b283\": container with ID starting with 9290649dd999dbcc3a55413835a677c7b4b0885da4d1227c19c2086069e9b283 not found: ID does not exist"
Sep 19 12:05:36.513015 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.512978    2559 scope.go:117] "RemoveContainer" containerID="ad763b6097f66b6838d3af3255275e859f8c885eee504ecb13d71bb4732c29bc"
Sep 19 12:05:36.513266 ip-10-0-140-136 kubenswrapper[2559]: E0919 12:05:36.513248    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad763b6097f66b6838d3af3255275e859f8c885eee504ecb13d71bb4732c29bc\": container with ID starting with ad763b6097f66b6838d3af3255275e859f8c885eee504ecb13d71bb4732c29bc not found: ID does not exist" containerID="ad763b6097f66b6838d3af3255275e859f8c885eee504ecb13d71bb4732c29bc"
Sep 19 12:05:36.513314 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.513271    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad763b6097f66b6838d3af3255275e859f8c885eee504ecb13d71bb4732c29bc"} err="failed to get container status \"ad763b6097f66b6838d3af3255275e859f8c885eee504ecb13d71bb4732c29bc\": rpc error: code = NotFound desc = could not find container \"ad763b6097f66b6838d3af3255275e859f8c885eee504ecb13d71bb4732c29bc\": container with ID starting with ad763b6097f66b6838d3af3255275e859f8c885eee504ecb13d71bb4732c29bc not found: ID does not exist"
Sep 19 12:05:36.517362 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.517343    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"]
Sep 19 12:05:36.520958 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.520940    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4a8a59-openshift-default-6759d6c794-v6qkl"]
Sep 19 12:05:36.525251 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.525232    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-workload-socket\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:36.525251 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.525251    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-credential-socket\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:36.525387 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.525261    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-podinfo\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:36.525387 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.525270    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-envoy\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:36.525387 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.525278    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istiod-ca-cert\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:36.525387 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.525286    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7672\" (UniqueName: \"kubernetes.io/projected/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-kube-api-access-k7672\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:36.525387 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.525293    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85-istio-token\") on node \"ip-10-0-140-136.ec2.internal\" DevicePath \"\""
Sep 19 12:05:36.595108 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:05:36.595081    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" path="/var/lib/kubelet/pods/6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85/volumes"
Sep 19 12:06:09.139306 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:06:09.139270    2559 scope.go:117] "RemoveContainer" containerID="48a899eb1ef58a9d454d0b75bf43751eca7b6d7ae7f2377eba6c484dbedfe61d"
Sep 19 12:10:09.194442 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:10:09.194365    2559 scope.go:117] "RemoveContainer" containerID="28fab2d9d61b04b44a0ea80580ddef297e8117e5b223d99eb4ed58a5a91eedb2"
Sep 19 12:10:09.206710 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:10:09.206684    2559 scope.go:117] "RemoveContainer" containerID="fd016c5993002d029ead51f0fbd7f2afd53c4da29f62aaac6ff01d983e73e1ac"
Sep 19 12:11:09.231096 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:11:09.231065    2559 scope.go:117] "RemoveContainer" containerID="3d2ed75d0b20390c89a0aaf1aa123e868601bcdf2d9f808061ee2053dac2598c"
Sep 19 12:11:09.241931 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:11:09.241907    2559 scope.go:117] "RemoveContainer" containerID="78cb2a7c3981d2aa7328aa8aa3c570b800f19887b9eec82389c16853eff98ab7"
Sep 19 12:12:09.264258 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:12:09.264229    2559 scope.go:117] "RemoveContainer" containerID="d0a43798d7661d7c8c9089ef00f549ed4cca83a99c86cbb3b9435d6ef72eec57"
Sep 19 12:12:48.206975 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:12:48.206937    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-54765d59fd-45g7g_c8344758-3571-45a9-bf9b-1c191dd357f5/manager/0.log"
Sep 19 12:12:48.332325 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:12:48.332291    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-78f7d99c47-22zfr_32f38755-5a54-4a59-a523-e8958e46b40f/manager/0.log"
Sep 19 12:12:49.231943 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:12:49.231911    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29830305-g7662_6af369db-935f-4496-8917-05fcd1a0deac/cleanup/0.log"
Sep 19 12:12:49.340821 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:12:49.340786    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29830320-xj475_3990acd7-d918-40ce-9c81-b1b708a07d8d/cleanup/0.log"
Sep 19 12:12:49.451545 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:12:49.451512    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-6l5wp_38cd5722-df7a-46c2-b49d-1875229db11c/postgres/0.log"
Sep 19 12:12:50.547516 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:12:50.547486    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-thdmv_3d1f84f0-c532-4981-aa15-dbed60d05e18/kuadrant-console-plugin/0.log"
Sep 19 12:12:51.384531 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:12:51.384490    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wkt7t_890b8972-ec70-408c-8846-fc41234aa2b4/discovery/0.log"
Sep 19 12:12:51.967985 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:12:51.967927    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-s8xn7_aa236dc9-323d-4d69-af0e-31787c8c5b24/payload-pre-processing/0.log"
Sep 19 12:12:52.089314 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:12:52.089288    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-cf2vb_a6292959-58d7-4e70-9bd2-ddc765c83de0/payload-processing/0.log"
Sep 19 12:12:52.675994 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:12:52.675964    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5d57fc946d-kjh7j_6d60fd1f-a2fd-485e-a8ad-d6e53a2b6642/main/0.log"
Sep 19 12:12:53.017538 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:12:53.017500    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6fl8v_96bbadaf-5cc8-499d-8754-835cb5eff26e/main/0.log"
Sep 19 12:13:01.369632 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:01.369543    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w6hmz_fdf9b2d4-b351-4020-8ee5-4b00b841066d/global-pull-secret-syncer/0.log"
Sep 19 12:13:01.460907 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:01.460871    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mxrb8_3c490d87-bfde-4aa7-b297-e71e3f1c8056/konnectivity-agent/0.log"
Sep 19 12:13:01.562858 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:01.562826    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-136.ec2.internal_8e524724c540c684f761004a0a6e1643/haproxy/0.log"
Sep 19 12:13:06.900949 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:06.900923    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-thdmv_3d1f84f0-c532-4981-aa15-dbed60d05e18/kuadrant-console-plugin/0.log"
Sep 19 12:13:08.323963 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.323914    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f/alertmanager/0.log"
Sep 19 12:13:08.345396 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.345363    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f/config-reloader/0.log"
Sep 19 12:13:08.364583 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.364560    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f/kube-rbac-proxy-web/0.log"
Sep 19 12:13:08.386442 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.386422    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f/kube-rbac-proxy/0.log"
Sep 19 12:13:08.406247 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.406225    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f/kube-rbac-proxy-metric/0.log"
Sep 19 12:13:08.425535 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.425515    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f/prom-label-proxy/0.log"
Sep 19 12:13:08.455982 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.455955    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6e64f5b2-dcd4-4333-ba77-3ddbdd0d5b6f/init-config-reloader/0.log"
Sep 19 12:13:08.526310 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.526292    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-sssb7_88bc1683-353e-4f02-ab36-8e5dce1dff76/kube-state-metrics/0.log"
Sep 19 12:13:08.543100 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.543081    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-sssb7_88bc1683-353e-4f02-ab36-8e5dce1dff76/kube-rbac-proxy-main/0.log"
Sep 19 12:13:08.566816 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.566792    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-sssb7_88bc1683-353e-4f02-ab36-8e5dce1dff76/kube-rbac-proxy-self/0.log"
Sep 19 12:13:08.599729 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.599669    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6f59cf996d-74dtw_204107b3-623b-4315-a136-1e895bed9f1d/metrics-server/0.log"
Sep 19 12:13:08.657433 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.657411    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-58dtg_e2b33905-4886-44b2-82e5-e9b07c2a9b46/node-exporter/0.log"
Sep 19 12:13:08.681230 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.681208    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-58dtg_e2b33905-4886-44b2-82e5-e9b07c2a9b46/kube-rbac-proxy/0.log"
Sep 19 12:13:08.702239 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.702219    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-58dtg_e2b33905-4886-44b2-82e5-e9b07c2a9b46/init-textfile/0.log"
Sep 19 12:13:08.908836 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.908750    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-v48hz_6efb3a91-0d2d-41f0-b169-9c6cda05cd2f/kube-rbac-proxy-main/0.log"
Sep 19 12:13:08.930620 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.930592    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-v48hz_6efb3a91-0d2d-41f0-b169-9c6cda05cd2f/kube-rbac-proxy-self/0.log"
Sep 19 12:13:08.963429 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:08.963396    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-v48hz_6efb3a91-0d2d-41f0-b169-9c6cda05cd2f/openshift-state-metrics/0.log"
Sep 19 12:13:09.216052 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:09.216027    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-97hh5_fea2f451-b87d-4f5d-a991-d3480ea0dc63/prometheus-operator/0.log"
Sep 19 12:13:09.236669 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:09.236642    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-97hh5_fea2f451-b87d-4f5d-a991-d3480ea0dc63/kube-rbac-proxy/0.log"
Sep 19 12:13:09.278043 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:09.278023    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5c6747dc7b-lmpmn_46543144-422d-406a-a81f-289c42535283/prometheus-operator-admission-webhook/0.log"
Sep 19 12:13:09.315334 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:09.315306    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-64d7fb9d6d-8pqcs_2b9901c8-0abe-466e-980d-67aafdfe730f/telemeter-client/0.log"
Sep 19 12:13:09.346487 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:09.346465    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-64d7fb9d6d-8pqcs_2b9901c8-0abe-466e-980d-67aafdfe730f/reload/0.log"
Sep 19 12:13:09.376609 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:09.376581    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-64d7fb9d6d-8pqcs_2b9901c8-0abe-466e-980d-67aafdfe730f/kube-rbac-proxy/0.log"
Sep 19 12:13:09.433153 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:09.433127    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f94cf4d78-26l2m_877d0665-b590-4c97-a37d-2c4672267afa/thanos-query/0.log"
Sep 19 12:13:09.470797 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:09.470735    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f94cf4d78-26l2m_877d0665-b590-4c97-a37d-2c4672267afa/kube-rbac-proxy-web/0.log"
Sep 19 12:13:09.510765 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:09.510744    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f94cf4d78-26l2m_877d0665-b590-4c97-a37d-2c4672267afa/kube-rbac-proxy/0.log"
Sep 19 12:13:09.560515 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:09.560497    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f94cf4d78-26l2m_877d0665-b590-4c97-a37d-2c4672267afa/prom-label-proxy/0.log"
Sep 19 12:13:09.597531 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:09.597510    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f94cf4d78-26l2m_877d0665-b590-4c97-a37d-2c4672267afa/kube-rbac-proxy-rules/0.log"
Sep 19 12:13:09.625141 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:09.625120    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f94cf4d78-26l2m_877d0665-b590-4c97-a37d-2c4672267afa/kube-rbac-proxy-metrics/0.log"
Sep 19 12:13:11.259820 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.259785    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"]
Sep 19 12:13:11.260386 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.260366    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" containerName="istio-proxy"
Sep 19 12:13:11.260440 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.260391    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" containerName="istio-proxy"
Sep 19 12:13:11.260440 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.260407    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76519908-fac0-4508-b417-78423417cf34" containerName="payload-pre-processing"
Sep 19 12:13:11.260440 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.260416    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="76519908-fac0-4508-b417-78423417cf34" containerName="payload-pre-processing"
Sep 19 12:13:11.260440 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.260434    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4afed85b-980b-4c82-b7ea-894e3aee992a" containerName="istio-proxy"
Sep 19 12:13:11.260570 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.260442    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4afed85b-980b-4c82-b7ea-894e3aee992a" containerName="istio-proxy"
Sep 19 12:13:11.260570 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.260454    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4afed85b-980b-4c82-b7ea-894e3aee992a" containerName="istio-proxy"
Sep 19 12:13:11.260570 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.260462    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4afed85b-980b-4c82-b7ea-894e3aee992a" containerName="istio-proxy"
Sep 19 12:13:11.260570 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.260492    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="010e5d08-cad9-43a9-9c7d-1041b58f24e7" containerName="revoke-keys"
Sep 19 12:13:11.260570 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.260500    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="010e5d08-cad9-43a9-9c7d-1041b58f24e7" containerName="revoke-keys"
Sep 19 12:13:11.260730 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.260578    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="010e5d08-cad9-43a9-9c7d-1041b58f24e7" containerName="revoke-keys"
Sep 19 12:13:11.260730 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.260586    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4afed85b-980b-4c82-b7ea-894e3aee992a" containerName="istio-proxy"
Sep 19 12:13:11.260730 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.260591    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b729bce-b9dc-4c2a-8ad7-1ad68b68dc85" containerName="istio-proxy"
Sep 19 12:13:11.260730 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.260598    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4afed85b-980b-4c82-b7ea-894e3aee992a" containerName="istio-proxy"
Sep 19 12:13:11.260730 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.260604    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="76519908-fac0-4508-b417-78423417cf34" containerName="payload-pre-processing"
Sep 19 12:13:11.263746 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.263730    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.265824 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.265800    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fgjdw\"/\"default-dockercfg-6b6xz\""
Sep 19 12:13:11.265824 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.265809    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fgjdw\"/\"openshift-service-ca.crt\""
Sep 19 12:13:11.266756 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.266731    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fgjdw\"/\"kube-root-ca.crt\""
Sep 19 12:13:11.269922 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.269816    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"]
Sep 19 12:13:11.309973 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.309952    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8968c2ab-5709-4ae8-98c6-d33c21600353-podres\") pod \"perf-node-gather-daemonset-hc69r\" (UID: \"8968c2ab-5709-4ae8-98c6-d33c21600353\") " pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.310093 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.309996    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8968c2ab-5709-4ae8-98c6-d33c21600353-proc\") pod \"perf-node-gather-daemonset-hc69r\" (UID: \"8968c2ab-5709-4ae8-98c6-d33c21600353\") " pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.310093 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.310024    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dwp8\" (UniqueName: \"kubernetes.io/projected/8968c2ab-5709-4ae8-98c6-d33c21600353-kube-api-access-9dwp8\") pod \"perf-node-gather-daemonset-hc69r\" (UID: \"8968c2ab-5709-4ae8-98c6-d33c21600353\") " pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.310172 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.310106    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8968c2ab-5709-4ae8-98c6-d33c21600353-lib-modules\") pod \"perf-node-gather-daemonset-hc69r\" (UID: \"8968c2ab-5709-4ae8-98c6-d33c21600353\") " pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.310172 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.310147    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8968c2ab-5709-4ae8-98c6-d33c21600353-sys\") pod \"perf-node-gather-daemonset-hc69r\" (UID: \"8968c2ab-5709-4ae8-98c6-d33c21600353\") " pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.411348 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.411323    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8968c2ab-5709-4ae8-98c6-d33c21600353-lib-modules\") pod \"perf-node-gather-daemonset-hc69r\" (UID: \"8968c2ab-5709-4ae8-98c6-d33c21600353\") " pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.411469 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.411357    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8968c2ab-5709-4ae8-98c6-d33c21600353-sys\") pod \"perf-node-gather-daemonset-hc69r\" (UID: \"8968c2ab-5709-4ae8-98c6-d33c21600353\") " pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.411469 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.411377    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8968c2ab-5709-4ae8-98c6-d33c21600353-podres\") pod \"perf-node-gather-daemonset-hc69r\" (UID: \"8968c2ab-5709-4ae8-98c6-d33c21600353\") " pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.411469 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.411410    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8968c2ab-5709-4ae8-98c6-d33c21600353-proc\") pod \"perf-node-gather-daemonset-hc69r\" (UID: \"8968c2ab-5709-4ae8-98c6-d33c21600353\") " pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.411469 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.411438    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dwp8\" (UniqueName: \"kubernetes.io/projected/8968c2ab-5709-4ae8-98c6-d33c21600353-kube-api-access-9dwp8\") pod \"perf-node-gather-daemonset-hc69r\" (UID: \"8968c2ab-5709-4ae8-98c6-d33c21600353\") " pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.411643 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.411471    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8968c2ab-5709-4ae8-98c6-d33c21600353-sys\") pod \"perf-node-gather-daemonset-hc69r\" (UID: \"8968c2ab-5709-4ae8-98c6-d33c21600353\") " pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.411643 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.411485    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8968c2ab-5709-4ae8-98c6-d33c21600353-proc\") pod \"perf-node-gather-daemonset-hc69r\" (UID: \"8968c2ab-5709-4ae8-98c6-d33c21600353\") " pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.411643 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.411519    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8968c2ab-5709-4ae8-98c6-d33c21600353-lib-modules\") pod \"perf-node-gather-daemonset-hc69r\" (UID: \"8968c2ab-5709-4ae8-98c6-d33c21600353\") " pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.411643 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.411519    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8968c2ab-5709-4ae8-98c6-d33c21600353-podres\") pod \"perf-node-gather-daemonset-hc69r\" (UID: \"8968c2ab-5709-4ae8-98c6-d33c21600353\") " pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.418715 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.418696    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dwp8\" (UniqueName: \"kubernetes.io/projected/8968c2ab-5709-4ae8-98c6-d33c21600353-kube-api-access-9dwp8\") pod \"perf-node-gather-daemonset-hc69r\" (UID: \"8968c2ab-5709-4ae8-98c6-d33c21600353\") " pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.575151 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.575075    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:11.847587 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:11.847561    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"]
Sep 19 12:13:11.849950 ip-10-0-140-136 kubenswrapper[2559]: W0919 12:13:11.849919    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8968c2ab_5709_4ae8_98c6_d33c21600353.slice/crio-7f944a03ccbade49cdab6caf928788336d1a20c8e8303d89993637342d34993e WatchSource:0}: Error finding container 7f944a03ccbade49cdab6caf928788336d1a20c8e8303d89993637342d34993e: Status 404 returned error can't find the container with id 7f944a03ccbade49cdab6caf928788336d1a20c8e8303d89993637342d34993e
Sep 19 12:13:12.843412 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:12.843372    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r" event={"ID":"8968c2ab-5709-4ae8-98c6-d33c21600353","Type":"ContainerStarted","Data":"d1f46fddd1fb1c7acd51cfc915d5b97ae6f445be015e16b714d3a92ea43918a9"}
Sep 19 12:13:12.843819 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:12.843428    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r" event={"ID":"8968c2ab-5709-4ae8-98c6-d33c21600353","Type":"ContainerStarted","Data":"7f944a03ccbade49cdab6caf928788336d1a20c8e8303d89993637342d34993e"}
Sep 19 12:13:12.843819 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:12.843535    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:12.859024 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:12.858979    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r" podStartSLOduration=1.858963766 podStartE2EDuration="1.858963766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:13:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:13:12.858338185 +0000 UTC m=+2344.780651063" watchObservedRunningTime="2026-09-19 12:13:12.858963766 +0000 UTC m=+2344.781276647"
Sep 19 12:13:13.664624 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:13.664590    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jkjln_aea6548e-5e35-48f8-b05b-813264f85e5d/dns/0.log"
Sep 19 12:13:13.682468 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:13.682445    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jkjln_aea6548e-5e35-48f8-b05b-813264f85e5d/kube-rbac-proxy/0.log"
Sep 19 12:13:13.746394 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:13.746342    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fjmwn_6ed690d2-ca31-447d-bda0-ee83ae9efc22/dns-node-resolver/0.log"
Sep 19 12:13:14.296723 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:14.296685    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2jpjr_4002479e-43d6-4fbb-9e74-f9a9d0c02c9c/node-ca/0.log"
Sep 19 12:13:16.121014 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:16.120980    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wkt7t_890b8972-ec70-408c-8846-fc41234aa2b4/discovery/0.log"
Sep 19 12:13:16.253818 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:16.253780    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-s8xn7_aa236dc9-323d-4d69-af0e-31787c8c5b24/payload-pre-processing/0.log"
Sep 19 12:13:16.293587 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:16.293558    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-cf2vb_a6292959-58d7-4e70-9bd2-ddc765c83de0/payload-processing/0.log"
Sep 19 12:13:17.238564 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:17.238530    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pz68l_3c4b91af-0fb0-4cf6-b0bf-08ef43e9b4d3/serve-healthcheck-canary/0.log"
Sep 19 12:13:17.793884 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:17.793852    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nfdpq_6d57e236-44c2-4acc-8ab0-342a59daf9c9/kube-rbac-proxy/0.log"
Sep 19 12:13:17.811370 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:17.811348    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nfdpq_6d57e236-44c2-4acc-8ab0-342a59daf9c9/exporter/0.log"
Sep 19 12:13:17.830596 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:17.830567    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nfdpq_6d57e236-44c2-4acc-8ab0-342a59daf9c9/extractor/0.log"
Sep 19 12:13:18.858379 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:18.858350    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fgjdw/perf-node-gather-daemonset-hc69r"
Sep 19 12:13:20.248170 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:20.248135    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-9f59f68b4-q2742_f75bdb2c-1911-4576-89d3-aec9a72b5d17/manager/0.log"
Sep 19 12:13:20.262493 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:20.262473    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-9f59f68b4-q2742_f75bdb2c-1911-4576-89d3-aec9a72b5d17/copy-manifests/0.log"
Sep 19 12:13:21.433164 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:21.433128    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wkt7t_890b8972-ec70-408c-8846-fc41234aa2b4/discovery/0.log"
Sep 19 12:13:21.527382 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:21.527347    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-s8xn7_aa236dc9-323d-4d69-af0e-31787c8c5b24/payload-pre-processing/0.log"
Sep 19 12:13:21.553428 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:21.553384    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-controller-manager-686b576b67-xwr22_e82cd179-7d1c-4d54-bb4b-5fe98b650efc/manager/0.log"
Sep 19 12:13:21.561013 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:21.560984    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-cf2vb_a6292959-58d7-4e70-9bd2-ddc765c83de0/payload-processing/0.log"
Sep 19 12:13:22.248444 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:22.248416    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-54765d59fd-45g7g_c8344758-3571-45a9-bf9b-1c191dd357f5/manager/0.log"
Sep 19 12:13:22.579361 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:22.579272    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-78f7d99c47-22zfr_32f38755-5a54-4a59-a523-e8958e46b40f/manager/0.log"
Sep 19 12:13:24.059025 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:24.058996    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wkt7t_890b8972-ec70-408c-8846-fc41234aa2b4/discovery/0.log"
Sep 19 12:13:24.176382 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:24.176355    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-s8xn7_aa236dc9-323d-4d69-af0e-31787c8c5b24/payload-pre-processing/0.log"
Sep 19 12:13:24.212573 ip-10-0-140-136 kubenswrapper[2559]: I0919 12:13:24.212538    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-cf2vb_a6292959-58d7-4e70-9bd2-ddc765c83de0/payload-processing/0.log"